Jump to content

NVMeFix

978b029efa981d618699b09868
 Share
103 posts in this topic

Recommended Posts

978b029efa981d618699b09868
978b029efa981d618699b09868
Posted
Posted

Today we are publishing NVMeFix, a set of patches for IONVMeController.

 

It currently only improves power management of non-Apple NVMe SSDs, but we will also be willing to fix other SSD incompatibility issues.

If you observe a third-party SSD misbehaving, feel free to provide as much information as you can at our bugtracker https://github.com/acidanthera/bugtracker/.

 

APST is short for Autonomous Power State Transition, a feature found in many modern SSDs. It allows the SSD to significantly reduce the power consumption when it detects the device has been idle for some time.

Apple NVM driver, IONVMeFamily, does support third-party SSDs, but only uses PCI power management for them, so power is not conserved unless the computer sleeps or hibernates. For Apple SSD controllers,

IONVMeFamily uses a vendor-specific command to switch to low-power mode.

 

NVMeFix attempts to auto-detect APST support and configure the controller accordingly. Unfortunately, some SSDs have broken APST support; in this case NVMeFix will try to disable it. You may inject 8 byte property

ps-max-latency-us=0 to the parent PCI device IODeviceTree entry in order to force disable APST. NVMeFix reports APST status via "apst" property of the IONVMeController in the IOService plane. Additionally, DEBUG

configuration of the kext will log the used APST tables.

 

The source code is available at https://github.com/acidanthera/NVMeFix. There is currently no builds available, as NVMeFix is only compatible with the latest, unreleased version of Lilu.

 

Note that besides APST and PCI power management, NVMe also supports host-managed power setting, which should decrease power usage even more. Unfortunately, we could not yet get it to work reliably yet: if

you're interested in technical details, let us know in the bugtracker.

You may also find this on real Macs if you are using a third-party SSD. The kext should be compatible with stock bootloader -- just install it and Lilu to /Library/Extensions.

 

Unfortunately, on Macs with the old NVMHCI DXE driver hibernation cannot be fixed from kernel side, as the bootloader hangs before it gets to the OS. This was at least the

case in my tests with full-disk encryption on; I don't know if boot gets any farther without encryption.

  • Like 9
  • Thanks 2
Link to comment
Share on other sites
Pavo
Pavo
Posted
Posted
3 hours ago, 978b029efa981d618699b09868 said:

Today we are publishing NVMeFix, a set of patches for IONVMeController.

 

It currently only improves power management of non-Apple NVMe SSDs, but we will also be willing to fix other SSD incompatibility issues.

If you observe a third-party SSD misbehaving, feel free to provide as much information as you can at our bugtracker https://github.com/acidanthera/bugtracker/.

 

APST is short for Autonomous Power State Transition, a feature found in many modern SSDs. It allows the SSD to significantly reduce the power consumption when it detects the device has been idle for some time.

Apple NVM driver, IONVMeFamily, does support third-party SSDs, but only uses PCI power management for them, so power is not conserved unless the computer sleeps or hibernates. For Apple SSD controllers,

IONVMeFamily uses a vendor-specific command to switch to low-power mode.

 

NVMeFix attempts to auto-detect APST support and configure the controller accordingly. Unfortunately, some SSDs have broken APST support; in this case NVMeFix will try to disable it. You may inject 8 byte property

ps-max-latency-us=0 to the parent PCI device IODeviceTree entry in order to force disable APST. NVMeFix reports APST status via "apst" property of the IONVMeController in the IOService plane. Additionally, DEBUG

configuration of the kext will log the used APST tables.

 

The source code is available at https://github.com/acidanthera/NVMeFix. There is currently no builds available, as NVMeFix is only compatible with the latest, unreleased version of Lilu.

 

Note that besides APST and PCI power management, NVMe also supports host-managed power setting, which should decrease power usage even more. Unfortunately, we could not yet get it to work reliably yet: if

you're interested in technical details, let us know in the bugtracker.

You may also find this on real Macs if you are using a third-party SSD. The kext should be compatible with stock bootloader -- just install it and Lilu to /Library/Extensions.

 

Unfortunately, on Macs with the old NVMHCI DXE driver hibernation cannot be fixed from kernel side, as the bootloader hangs before it gets to the OS. This was at least the

case in my tests with full-disk encryption on; I don't know if boot gets any farther without encryption.

Does not work 

2020-01-02 15:37:50.677072-0500  localhost kernel[0]: Couldn't alloc class "NVMeFix"
2020-01-02 15:37:50.679072-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2020-01-02 15:37:50.679075-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> ReleaseIDNode
2020-01-02 15:37:50.679077-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.80.1/IONVMeController.cpp
2020-01-02 15:37:50.679081-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> line: 5416
2020-01-02 15:37:50.679083-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> 
2020-01-02 15:37:50.679959-0500  localhost kernel[0]: (IOAHCIFamily) <IOAHCIFamily`IOAHCIPortMultiplierGlobals::IOAHCIPortMultiplierGlobals()> [AHCI][PML][00000000]+IOAHCIPortMultiplierGlobals::IOAHCIPortMultiplierGlobals
2020-01-02 15:37:50.679960-0500  localhost kernel[0]: (IOAHCIFamily) <IOAHCIFamily`IOAHCIPortMultiplierGlobals::IOAHCIPortMultiplierGlobals()> [AHCI][PML][00000000]-IOAHCIPortMultiplierGlobals::IOAHCIPortMultiplierGlobals
2020-01-02 15:37:50.680309-0500  localhost kernel[0]: <compose failure [UUID]>

 

Link to comment
Share on other sites
978b029efa981d618699b09868
978b029efa981d618699b09868
Posted
  • Author
Posted
2 hours ago, Pavo said:

Does not work 


2020-01-02 15:37:50.677072-0500  localhost kernel[0]: Couldn't alloc class "NVMeFix"
2020-01-02 15:37:50.679072-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2020-01-02 15:37:50.679075-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> ReleaseIDNode
2020-01-02 15:37:50.679077-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.80.1/IONVMeController.cpp
2020-01-02 15:37:50.679081-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> line: 5416
2020-01-02 15:37:50.679083-0500  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> 
2020-01-02 15:37:50.679959-0500  localhost kernel[0]: (IOAHCIFamily) <IOAHCIFamily`IOAHCIPortMultiplierGlobals::IOAHCIPortMultiplierGlobals()> [AHCI][PML][00000000]+IOAHCIPortMultiplierGlobals::IOAHCIPortMultiplierGlobals
2020-01-02 15:37:50.679960-0500  localhost kernel[0]: (IOAHCIFamily) <IOAHCIFamily`IOAHCIPortMultiplierGlobals::IOAHCIPortMultiplierGlobals()> [AHCI][PML][00000000]-IOAHCIPortMultiplierGlobals::IOAHCIPortMultiplierGlobals
2020-01-02 15:37:50.680309-0500  localhost kernel[0]: <compose failure [UUID]>

 

I can't conclude that based on this log. The "couldn't alloc class" error is supposed to be there. You need to check if there's "apst" property for your NVM controller in ioreg

  • Like 1
Link to comment
Share on other sites
978b029efa981d618699b09868
978b029efa981d618699b09868
Posted
  • Author
Posted (edited)

Since commit 3d4217bf there is now support for host-driven power management to transition between operational power states. It works by having IOKit notify us about (lack of) controller activity within certain time intervals. This should integrate transparently with PCI link power management and APST. I'm not entirely sure which timer interval is useful; you may experiment with it here https://github.com/acidanthera/NVMeFix/blob/master/NVMeFix/nvme_pm.cpp#L109 (in seconds). This is still not as advanced as power management for Apple controllers: they implement Quality of Service, where clients indicate the expected workloads and operation priorities, and power state is selected accordingly.

 

You may check the supported power states yourself by using smartmontools. For example, for my controller, it reports

the following states: 

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     9.00W       -        -    0  0  0  0        0       0
 1 +     4.60W       -        -    1  1  1  1        0       0
 2 +     3.80W       -        -    2  2  2  2        0       0
 3 -   0.0450W       -        -    3  3  3  3     2000    2000
 4 -   0.0040W       -        -    4  4  4  4     6000    8000

NVMeFix then uses the first three states for host-driven power management, and latter, non-operational states, are used for APST. IOPower:/IOPowerConnection/IOPMrootDomain/IOPowerConnection/NVMePMProxy IOPowerManagement dictionary will report four power states possible: state 0 corresponds to power off state and should not be reachable; state 1 corresponds to state 2 in the above report etc.

Edited by 978b029efa981d618699b09868
  • Like 1
Link to comment
Share on other sites
tmbt
tmbt
Posted
Posted (edited)

Hi,

first of all thanks for this new kext because my laptop's battery needed it badly.


I successfully compiled it and i'm using it right now. The kext is working because finally using Intel Power Gadget PKG is down to 0.8W as it was when i was using a normal SSD. Before this kext using my new NVME m.2 drive i had 1.8W on idle. But it seems power management doesn't work after wake from sleep because after waking up the laptop power consumption is back to 1.8W on idle. 

I'm not an expert but if someone could point me on the right direction to solve this issue or better analyze it please let me know.

Should i open an issue on github project page ?

 

 

Thanks

Mattia

 

Edited by tmbt
Link to comment
Share on other sites
978b029efa981d618699b09868
978b029efa981d618699b09868
Posted
  • Author
Posted (edited)
22 hours ago, tmbt said:

Hi,

first of all thanks for this new kext because my laptop's battery needed it badly.


I successfully compiled it and i'm using it right now. The kext is working because finally using Intel Power Gadget PKG is down to 0.8W as it was when i was using a normal SSD. Before this kext using my new NVME m.2 drive i had 1.8W on idle. But it seems power management doesn't work after wake from sleep because after waking up the laptop power consumption is back to 1.8W on idle. 

I'm not an expert but if someone could point me on the right direction to solve this issue or better analyze it please let me know.

Should i open an issue on github project page ?

 

 

Thanks

Mattia

 

This should be fixed in master (untested). You may look at kernel log with DEBUG build to check if APST is re-enabled at wake.

 

Some documentation has been added to the public repo. If things go well, the kext is to be released together with Lilu soon.

Edited by 978b029efa981d618699b09868
  • Like 1
Link to comment
Share on other sites
ivan_htp
ivan_htp
Posted
Posted (edited)

I have problems with IONVME controller since I'd assembled my Hackintosh. High sierra, mojave, catalina.. it aways gives kernel panics after some usage time and it's always the same log: 

panic(cpu 0 caller 0xffffff7f93af44a7): nvme: "Fatal error occurred. CSTS=0xffffffff US[1]=0x0 US[0]=0x206 VID=0x144d DID=0xa804
. FW Revision=3B7QCXE7\n"@/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.40.4/IONVMeController.cpp:5258
Backtrace (CPU 0), Frame : Return Address
0xffffff820597b9e0 : 0xffffff8011139a3b 
0xffffff820597ba30 : 0xffffff8011270fe5 
0xffffff820597ba70 : 0xffffff8011262a5e 
0xffffff820597bac0 : 0xffffff80110e0a40 
0xffffff820597bae0 : 0xffffff8011139127 
0xffffff820597bbe0 : 0xffffff801113950b 
0xffffff820597bc30 : 0xffffff80118d17f9 
0xffffff820597bca0 : 0xffffff7f93af44a7 
0xffffff820597bcc0 : 0xffffff7f93adf47a 
0xffffff820597be20 : 0xffffff8011844f79 
0xffffff820597be90 : 0xffffff8011844e99 
0xffffff820597bec0 : 0xffffff801117b625 
0xffffff820597bf40 : 0xffffff801117b151 
0xffffff820597bfa0 : 0xffffff80110e013e 
      Kernel Extensions in backtrace:
         com.apple.iokit.IONVMeFamily(2.1)[B8EE9C14-90EE-39E2-8565-0B00AC09499B]@0xffffff7f93ad1000->0xffffff7f93b13fff
            dependency: com.apple.driver.AppleMobileFileIntegrity(1.0.5)[180FC5AB-0E47-35BC-91DF-609EBE8ED4A5]@0xffffff7f91def000
            dependency: com.apple.iokit.IOPCIFamily(2.9)[AA7C7A4F-9F5D-3533-9E78-177C3B6A72BF]@0xffffff7f91ea2000
            dependency: com.apple.driver.AppleEFINVRAM(2.1)[A86937B2-6177-3699-BE65-6762DDC120C6]@0xffffff7f92094000
            dependency: com.apple.iokit.IOStorageFamily(2.1)[A9967DFD-FCB3-3536-B071-F8B46494895B]@0xffffff7f91a65000
            dependency: com.apple.iokit.IOReportFamily(47)[2CCA7DD3-C33F-3CA4-A213-BC83D3D997B0]@0xffffff7f91a9f000

BSD process name corresponding to current thread: kernel_task
Boot args: dart=0

I've seen a lot of reports like these on Samsung nvme ssds. some people say its temperature, others power management, others say that 3rd parties drives are not supported at all. 

My question is: there is any chance of this fix to solve this problem? Even if it's unsure, is there any risk of trying it?
These crashes take my peace away on the last two years and I would replace them if nvmes weren't so expensive in brazil :(

 

Edited by ivan_htp
Link to comment
Share on other sites
978b029efa981d618699b09868
978b029efa981d618699b09868
Posted
  • Author
Posted

I suspect this issue has been fixed by https://patchwork.kernel.org/patch/10542393/. We may look into porting the patch.

 

Can you submit a detailed log to bugtracker in the meantime? Make sure to include "keepsyms=1" in your boot-args. You should also submit kernel log before the panic if you manage to get it.

  • Like 1
Link to comment
Share on other sites
  • 2 weeks later...
cele_82
cele_82
Posted
Posted

I have a Dell XPS 9360 with PC300 NVMe SK hynix 256GB...which isn't power managed and makes the laptop quite warm and limit its battery life...

 

I've installed the NVMeFix.kext in kexts/Other of my EFI/CLOVER folder. Looking with IORegistry I can see apst set to TRUE but no changes in termperature or power reported by the Intel Power Gadget. 

 

 

Attached my IORegistry dump... anything else I can try ? I would love to power manage the NVME...

 

 

 

Thanks again for all your efforts @978b029efa981d618699b09868

forumusername.ioreg.txt

Link to comment
Share on other sites
978b029efa981d618699b09868
978b029efa981d618699b09868
Posted
  • Author
Posted
On 1/26/2020 at 9:01 PM, cele_82 said:

I have a Dell XPS 9360 with PC300 NVMe SK hynix 256GB...which isn't power managed and makes the laptop quite warm and limit its battery life...

 

I've installed the NVMeFix.kext in kexts/Other of my EFI/CLOVER folder. Looking with IORegistry I can see apst set to TRUE but no changes in termperature or power reported by the Intel Power Gadget. 

 

 

Attached my IORegistry dump... anything else I can try ? I would love to power manage the NVME...

 

 

 

Thanks again for all your efforts @978b029efa981d618699b09868

forumusername.ioreg.txt

If you run the debug build with -nvmefdbg boot arg, what APST values does it log?

There's not much we can do if the controller is misbehaving. Can you test what results you're getting on Windows or Linux? If using Windows, make sure you're using StorNVMe (the default driver by Microsoft)

  • Like 1
Link to comment
Share on other sites
cele_82
cele_82
Posted
Posted
10 hours ago, 978b029efa981d618699b09868 said:

If you run the debug build with -nvmefdbg boot arg, what APST values does it log?

There's not much we can do if the controller is misbehaving. Can you test what results you're getting on Windows or Linux? If using Windows, make sure you're using StorNVMe (the default driver by Microsoft)

 

You mean in syslog ? I can find that out.

 

If you tell me what you need from Ubuntu I'll get it for you, and thanks again for starting to look at this problem.

Link to comment
Share on other sites
hackattak
hackattak
Posted
Posted (edited)

Anyone using this with the Sabrent Rocket 2TB?

 

I have one of those I have been using in my desktop Hack that I might transition to a 2015 15” MBP if it is deemed to be a great drive for that usage. 
 

Thank you

Edited by TurbineSeaplane
Link to comment
Share on other sites
978b029efa981d618699b09868
978b029efa981d618699b09868
Posted
  • Author
Posted
On 1/28/2020 at 10:29 PM, cele_82 said:

 

 

@978b029efa981d618699b09868 here's the output of dmesg from Mojave... let me know what you need from ubuntu

dmesg.txt

I don't see anything useful in the log. You should use syslog instead and look for messages starting with "apst @" and " nvmef @".

On Linux you should check your power consumption to see if APST does anything there. The driver for Linux can produce some logs, but you have to figure out how to find and read them on your own.

  • Like 1
Link to comment
Share on other sites
cele_82
cele_82
Posted
Posted
On 1/29/2020 at 10:56 PM, 978b029efa981d618699b09868 said:

I don't see anything useful in the log. You should use syslog instead and look for messages starting with "apst @" and " nvmef @".

On Linux you should check your power consumption to see if APST does anything there. The driver for Linux can produce some logs, but you have to figure out how to find and read them on your own.

 

Nothing in /var/log/system.log for "apst" or "nvmef" (I searched case insensitive, only stuff is in dimesg)

 

Not sure what do to next.

Link to comment
Share on other sites
978b029efa981d618699b09868
978b029efa981d618699b09868
Posted
  • Author
Posted (edited)
On 2/3/2020 at 9:56 AM, cele_82 said:

will try, I've also used the Console app. Can see the log but nothing for those entries you mentioned.

Console doesn't search for old records. You need to invoke something like

  

log show --style syslog --last boot --predicate  "process == \"kernel\" and eventMessage CONTAINS[c] \"nvme\""

Search for the APST values set by NVMeFix, and compare them with the values on Linux (I think nvme-cli can do that, google it). Also, measure and compare battery life on Linux with Mac.

Edited by 978b029efa981d618699b09868
  • Like 1
Link to comment
Share on other sites
 Share
Go to topic listing
×
×
  • Create New...