tktech / smartie Goto Github PK

View Code? Open in Web Editor NEW

9.0 4.0 3.0 897 KB

Pure-python ATA/SATA/ATAPI/SCSI and disk enumeration library for Linux/Windows/OS X.

Home Page: https://tkte.ch/smartie/

License: MIT License

Python 100.00%

ata atapi python sata scsi

smartie's Introduction

SMARTie

This is a pure-python, 0-dependency library for getting basic disk information such as model, serial number, disk health, temperature, and SMART data. It supports both SCSI/ATA and NVMe devices.

Documentation

Read the getting started guide and API documentation at https://tkte.ch/smartie/.

Installation

SMARTie currently requires Python 3.8 or greater.

pip install smartie

If you want the command line tools, you'll also want to do:

pip install smartie[cli]

FAQ

This library isn't returning any of my drives?

The APIs this library uses to communicate with devices typically require root (on Linux) or administrator (on Windows) access to work.

My drive doesn't work with this library?

Support for drives that don't follow modern standards is still a work in progress. Open an issue.

Library Y does X, can I copy that code?

It depends. This library is available under the MIT license and is a fun side project. I want anyone to be able to use it. Many existing projects are GPL or LGPL, so you need to avoid them when contributing to this project. Instead:

Use the specifications or vendor documentation whenever possible.
Use the SG_IO documentation by Danny (https://sg.danny.cz/sg/).
Use the conversations in mailing lists and bug trackers, while avoiding the code.

Does this library support RAID controllers?

Some RAID/HBA controllers are known to work and others should as well. If you're using a RAID controller, and it doesn't work, open an issue.

RAID/HBA Adapter	OS under test	SCSI Supported	ATA Supported
Broadcom RAID 9440	CentOS 8.4	Yes	Yes
Broadcom RAID 9560	CentOS 8.4	Yes	Yes
Broadcom HBA 9500	CentOS 8.4	Yes	Yes
ThinkSystem 930	CentOS 8.4	Yes	Yes
ThinkSystem 940	CentOS 8.4	Yes	Yes
ThinkSystem 4350	CentOS 8.4	Yes	Yes
ThinkSystem 5350	CentOS 8.4	Yes	Yes

Only INQUIRY, IDENTIFY and SMART-related commands are tested, and the tests use the latest driver from vendor support web. If it does not work in your environment, open an issue. The compatibility should be same as pydiskcmd. Find the testing methodology here.

ATA, ATAPI, SCSI, NVMe, what?

Acronyms, acronyms everywhere! What does any of this mean?

ATA: Advanced Technology Attachment.
ATAPI: AT Attachment Packet Interface.
SCSI: Small Computer System Interface.
NVMe: Non-Volatile Memory Express. The standard for connecting "modern" solid-state drives to a computer, typically through PCI-e.
SATA: Serial ATA.
PATA: Parallel ATA.
S.M.A.R.T: Self-Monitoring, Analysis, and Reporting Technology. A standard for hard drives and solid-state drives to report their health and status.

smartie's People

Contributors

Stargazers

Watchers

Forkers

jackeichen mewmix zakdanger

smartie's Issues

Help to test this tool in HBA/RAID

I can help to test this tool with HBA/RAID controllers. I think this code may support disk thourgh HBA and JBOD Mode through RAID.

I will attach the test result in future.

says to install via pip, but pip version is v1.0.3

Can you please put the latest version v2.1.0 on pip so that it can be easly installed?

Implement FreeBSD Support.

We should be able to use the pass driver to support SCSI/ATA commands on FreeBSD.

get_all_devices support
SCSI/ATA support

Reference: https://www.freebsd.org/cgi/man.cgi?query=pass&sektion=4&manpath=freebsd-release-ports

Implement BeOS/Haiku support

Haiku supports all of the interfaces we require for basic support, but has little documentation around it. IRC channel and community are very helpful.

get_all_devices support
SCSI/ATA support
NVMe support (may not currently be possible given the state of the Haiku driver)

SCSI Status not checked or made available after SCSI Command

Windows and Linux both support returning a status value for the scsi device after a scsi command is attempted.
This is separate to the sense data and is not currently handled by smartie.

On Linux this value comes from 'SGIOHeader.status'.
On Windows this value comes from 'SCSIPassThroughDirect.scsi_status'

The values returned by Windows and Linux are different, but they could be mapped onto matching values when returned by smartie. It looks like you aready have the Linux values in "scsi/structures.py" under "class StatusCode(enum.IntEnum):"

class StatusCode(enum.IntEnum):
    """
    The possible values for `SGIOHeader.status`.
    """

    GOOD = 0x00
    CHECK_CONDITION = 0x01
    CONDITION_GOOD = 0x02
    BUSY = 0x04
    INTERMEDIATE_GOOD = 0x08
    INTERMEDIATE_C_GOOD = 0x0A
    RESERVATION_CONFLICT = 0x0C

The values for Windows can be gotten from:
https://github.com/9176324/WinDDK/blob/master/7600.16385.1/inc/ddk/scsi.h
They are:

//
// SCSI bus status codes.
//

#define SCSISTAT_GOOD                  0x00
#define SCSISTAT_CHECK_CONDITION       0x02
#define SCSISTAT_CONDITION_MET         0x04
#define SCSISTAT_BUSY                  0x08
#define SCSISTAT_INTERMEDIATE          0x10
#define SCSISTAT_INTERMEDIATE_COND_MET 0x14
#define SCSISTAT_RESERVATION_CONFLICT  0x18
#define SCSISTAT_COMMAND_TERMINATED    0x22
#define SCSISTAT_QUEUE_FULL            0x28

If you are already making changes to 'issue_command()' to return a tuple that now includes the data size, perhaps you could also add the scsi status to the tuple.

Implement OS X Support

OS X provides limited support for SCSI/ATA via IOKit. We can implement support for IDENTIFY, INQUIRY (maybe), and SMART-related commands.

get_all_devices support
SCSI/ATA support
NVMe support (difficult, don't have access to an NVMe macbook)

Reference: https://developer.apple.com/documentation/iokit

Got error when connect a "USB to SATA" device

I have a "USB to SATA" device(and connect a SATA SSD to it) in system, and install CentOS 8.4 to it.

When I use it as follow:

from smartie.device import get_all_devices

for device in get_all_devices():
    print(device.path)
    print(device.model)
    print(device.serial)
    print(device.temperature)

Error output:

/dev/sda
Traceback (most recent call last):
  File "/root/my_test.py", line 6, in <module>
    print (device.model)
  File "/usr/local/lib/python3.9/site-packages/smartie/scsi/__init__.py", line 133, in model
    identity, sense = self.identify()
  File "/usr/local/lib/python3.9/site-packages/smartie/scsi/__init__.py", line 115, in identify
    sense = self.issue_command(Direction.FROM, command16, identity)
  File "/usr/local/lib/python3.9/site-packages/smartie/scsi/linux.py", line 67, in issue_command
    raise OSError(ctypes.get_errno())
OSError: 25

Implement NVME support on Linux

We don't currently support nvme drives on Linux. We should definitely be able to do this, just needs investigation.

smartie cannot support SAS Disk

I test smartie with SAS disk, here is my disk info:

[root@localhost ~]# smartctl -a /dev/sdb
smartctl 7.1 2020-04-05 r5049 [x86_64-linux-4.18.0-305.3.1.el8.x86_64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               XXXXXXX ( I hide information here)
Product:              MZILT1T9HBJRV3
Revision:             CN32
Compliance:           SPC-4
User Capacity:        1,920,383,410,176 bytes [1.92 TB]
Logical block size:   512 bytes
Physical block size:  4096 bytes
LU is resource provisioned, LBPRZ=1
Rotation Rate:        Solid State Device
Form Factor:          2.5 inches
Logical Unit id:      XXXXXXX ( I hide information here)
Serial number:        XXXXXXX ( I hide information here)
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Fri Sep  1 07:52:12 2023 EDT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK

Percentage used endurance indicator: 0%
Current Drive Temperature:     31 C
Drive Trip Temperature:        65 C

Manufactured in week 41 of year 2019
Accumulated start-stop cycles:  145
Specified load-unload count over device lifetime:  0
Accumulated load-unload cycles:  0
Elements in grown defect list: 0

Error counter log:
           Errors Corrected by           Total   Correction     Gigabytes    Total
               ECC          rereads/    errors   algorithm      processed    uncorrected
           fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
read:          0        0         0         0          0      44609.083           0
write:         0        0         0         0          0      80920.541           0
verify:        0        0         0         0          0          0.109           0

Non-medium error count:      828

SMART Self-test log
Num  Test              Status                 segment  LifeTime  LBA_first_err [SK ASC ASQ]
     Description                              number   (hours)
# 1  Background short  Completed                   -     819                 - [-   -    -]

Long (extended) Self-test duration: 3600 seconds [60.0 minutes]

Error occurs when use smartie details:

[root@localhost ~]# smartie details /dev/sdb
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/smartie/scsi/__init__.py", line 115, in identify
    sense = self.issue_command(Direction.FROM, command16, identity)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/smartie/scsi/linux.py", line 69, in issue_command
    return self.parse_sense(raw_sense.raw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/smartie/scsi/__init__.py", line 53, in parse_sense
    raise SenseError(sense.sense_key, sense=sense)
smartie.errors.SenseError: <SenseError(error_code=0x05, err='Illegal Request')>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/smartie", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/smartie/cli.py", line 130, in details_command
    details_table.add_row("Model Number", device.model)
                                          ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/smartie/scsi/__init__.py", line 133, in model
    identity, sense = self.identify()
                      ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/smartie/scsi/__init__.py", line 124, in identify
    sense = self.issue_command(Direction.FROM, command16, identity)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/smartie/scsi/linux.py", line 69, in issue_command
    return self.parse_sense(raw_sense.raw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/smartie/scsi/__init__.py", line 53, in parse_sense
    raise SenseError(sense.sense_key, sense=sense)
smartie.errors.SenseError: <SenseError(error_code=0x05, err='Illegal Request')>

SCSI cmd requested data size can be different to actual data size transferred. No way to see this atm.

When doing scsi commands that request for data from a device, the actual size of the data transferred can be different to the requested size. I guess this usually occurs when you ask for a big amount of data, but the device only supports a smaller amount of data.

see: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddscsi/ns-ntddscsi-_scsi_pass_through_direct
"The value in DataTransferLength must be an integral multiple of this predefined, minimum length that is specified by the device. If an underrun occurs, the miniport driver must update this member to the number of bytes actually transferred."

An example of this is when doing an Inquiry (12h) command.
On a dvd-drive I have I ask for 0xF0 bytes but my device only returns 0x60 bytes.
If you aren't made aware that the actual data returned is less than the requested data, you may then try to use the full amount of data which is not going to be the correct or expected values.
(Note that the amount of data returned by an Inquiry command can be device specific. So it may not be 0x60 bytes on all devices)

For Windows the value "SCSIPassThroughDirect.data_transfer_length" gets updated during the call to DeviceIoControl() to contain the actual size of the data transferred. So in my above example it would be intially set to 0xF0 bytes before calling DeviceIoControl(), then once the call is finished, reading out the value would show that it is now 0x60.

It looks like SGIO on Linux has a value "resid" to show the actual number of bytes transferred on it.

I am not sure how to best handle this in your smartie code.
Do you think it would be safe to truncate the 'data' parameter passed into SCSIDevice.issue_command()?
Or instead return 2 values, where one is the sense data and the other is the data?
Or include a new parameter passed in for data size?