Giter Club home page Giter Club logo

rootonnvme's Introduction

rootOnNVMe

Switch the rootfs to a NVMe SSD on the Jetson Xavier NX and Jetson AGX Xavier

These scripts install a service which runs at startup to point the rootfs to a SSD installed on /dev/nvme0 (the M.2 Key M slot).

This is taken from the NVIDIA Jetson AGX Xavier forum https://forums.developer.nvidia.com/t/how-to-boot-from-nvme-ssd/65147/22, written by user crazy_yorik (https://forums.developer.nvidia.com/u/crazy_yorick). Thank you crazy_yorik!

This procedure should be done on a fresh install of the SD card using JetPack 4.3+. Install the SSD into the M.2 Key M slot of the Jetson, and format it gpt, ext4, and setup a partition (p1). The AGX Xavier uses eMMC, the Xavier NX uses a SD card in the boot sequence.

Next, copy the rootfs of the eMMC/SD card to the SSD

$ ./copy-rootfs-ssd.sh

Then, setup the service. This will copy the .service file to the correct location, and install a startup script to set the rootfs to the SSD.

$ ./setup-service.sh

After setting up the service, reboot for the changes to take effect.

Boot Notes

These script changes the rootfs to the SSD after the kernel image is loaded from the eMMC/SD card. For the Xavier NX, you will still need to have the SD card installed for booting. As of this writing, the default configuration of the Jetson NX does not allow direct booting from the NVMe.

Upgrading

Once this service is installed, the rootfs will be on the SSD. If you upgrade to a newer version of L4T using OTA updates (using the NVIDIA .deb repository), you will need to also apply those changes to the SD card that you are booting from.

Typically this involves copying the /boot* directory and /lib/modules/<kernel name>/ from the SSD to the SD card. If they are different, then modules load will be 'tainted', that is, the modules version will not match the kernel version.

Notes

  • Initial Release, May 2020
  • JetPack 4.4 DP
  • Tested on Jetson Xavier NX

rootonnvme's People

Contributors

jetsonhacks avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rootonnvme's Issues

[BUG]

Describe the issue
When I used this: rootOnNVMe then when I move the system from emmc to NVMe my USB was influenced... When I plug my GNSS receivers they could not be detected, like the command: ls /dev/tty*, there was no USB detected like: /dev/ttyACM0 or /dev/ttyUSB0. Do you know if there is anything I can do to solve this issue?

What version of L4T/JetPack
L4T/JetPack version: Jetpack 4.6.3

Which Jetson
Jetson: Xavier NX

To Reproduce
Steps to reproduce the behavior:
./copy-rootfs-ssd.sh and ./setup-service.sh

Expected behavior
Jetson Xavier NX will boot with NVMe and the USB can work normally.

Additional context
Add any other context about the problem here.

Wonderful, works on the Nvidia Jetson AGX Xavier with PCIe NVMe

Great piece of code; I just tried that out on a Xavier AGX 32Gb with

  • microSD boot card (flashed with current jetpack), not touching the built-in storage
  • PCIe -adapter with NVNe 1TB SSD

I noticed that the NVMe showed up as /dev/nvme0n1 in its factory configuration, and I left it like that. I then modified your scripts to this device (ext4, whole device; did not check what kind of partition table Samsung put here). I can confirm that everything works on the AGX. Thanks again!

Is it right to modify /etc/fstab?

Thanks for your sharing!
First, I format my SSD and create partition on it.
After that, I follow your instruction in the README.md file to run those 2 shell scripts.

However, after reboot, the operating system was timed out waiting for some device.
The followings were come out:

  • Dependency failed for /media/ssd
  • Dependency failed for Local File Systems.

The jetson Xavier NX entered emergency mode and it could not recover to default mode.
Finally, I edited "/etc/fstab"

There are two rows inside this file.

/dev/root / ext4 defaults 0 1
UUID=ae9dc... /media/ssd ext4 defaults 0 0

My solution is to comment out the rows with UUID. After that, everything is worked and I can use the SSD as my root file system.
Is it really a right choice to modify /etc/fstab.

/media/ssd is the default directory of my ssd when I was using sd card as my directory. I think this directory is no longer existed after SSD became the root file system of my jetson xavier NX.

Jetson ORIN: librealsense SDK is not detecting camera d435I, rules are not loading while root from SDCARD

Describe the issue
Hi, We are using jetson orin while booting from sdcard. We have installed ROS2 humble docker with realsense, as mentioned by https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/tree/main. Our docker chain is humble.nav2.realsense.

While running inside docker, realsense-viewer is not detecting the camera attached to Jetson Orin. we copied 99-realsense-libusb.rules from https://github.com/IntelRealSense/librealsense/blob/master/config/99-realsense-libusb.rules.

What we observed is while running sudo udevadm control --reload-rules showing us error log as Running in chroot, ignoring request. Is this a cause of not detecting camera ?

For the reference while creating setssdroot.service our CHROOT_PATH=/sdcard in the file https://github.com/jetsonhacks/rootOnNVMe/blob/master/data/setssdroot.sh if it makes any impact.

What version of L4T/JetPack
L4T: 34.1
Jetpack: 5.0.1

Which Jetson
Jetson: Agx Orin

Do anyone have idea working realsense camera inside humble docker ?

Error: cannot create regular file '/mnt/etc/systemd/system/setssdroot.service'

Hi there,
I used this repo to boot Jetson AGX Xavier from the newly installed NVMe. At the first attempt, I had successfully finished all the steps, and it booted from the NVMe thereafter. Few days later, it was, however, not booting for some reasons. Therefore, I used the same steps from this repo to reinstall the os to NVMe, but ended up the following error:

cp: cannot create regular file '/mnt/etc/systemd/system/setssdroot.service': No such file or directory
cp: cannot create regular file '/mnt/sbin/setssdroot.sh': No such file or directory
Service to set the rootfs to the SSD installed.
Make sure that you have copied the rootfs to SSD.
Reboot for changes to take effect.

It looks like to me, the NVMe was not mounted properly

What version of L4T/JetPack
L4T/JetPack version: 4.4.x

Which Jetson
Jetson: NVIDIA Jetson AGX Xavier Developer Kit

To Reproduce
Steps to reproduce the behavior:

cd rootOnNVMe
./setup-service.sh

Expected behavior
./setup-service.sh command should copy all the required system file to NVMe and create a service to boot from the NVMe.

Docs - Upgrading & modules, etc

The current documentation states:

Once this service is installed, the rootfs will be on the SSD. If you upgrade to a newer version of L4T using OTA updates (using the NVIDIA .deb repository), you will need to also apply those changes to the SD card that you are booting from.

Typically this involves copying the /boot* directory and /lib/modules// from the SSD to the SD card. If they are different, then modules load will be 'tainted', that is, the modules version will not match the kernel version.

Can someone comment on exactly what files/etc are loaded from the base boot device? Clearly the kernel is loaded from the internal boot device. However, am I right in understanding from the above that all kernel modules are loaded from the internal boot device as well?

Also: After booting with the nvme hack, is it safe to mount the internal boot device and write to it (ex to update said modules) before rebooting?

Thank you!

AGX Xavier SSD

L4T/JetPack version: 4.6
Jetson: XavierAGX

I have a question for this installation. I installed without any issue but now, i have two ubuntu files on eMMC and on SSD. If i delete all files on eMMC, will i get any trouble?

[BUG]USB mouse and keyboard not functioning after reboot

General USB mouse and keyboard(both not wireless).

I could boot from SD and the mouse/keyboard were functioning normally.
After rootOnNVMe, the mouse and keyboard are not functioning at boot screen. I have to unplug the mouse and keyboard and plug them back again to enable them.

[BUG] Sometimes boot from sd card

I used your code to make the Jetson Xavier NX boot from 128GB SSD successfully.
However, the jetson sometimes boots from sd card randomly. It can boot from SSD again after reboot.

Did someone meet this problem before?

[BUG]

Describe the issue
Hi, I want to use USB device on ACM device. But when I plug-in USB, it didn't show up the ttyACM0 in /dev.
I tested on SD card without SSD, it will show up.

What version of L4T/JetPack
L4T/JetPack version: 4.4.1

Which Jetson
Jetson: NX

With SSD and use rootOnNVMe
[ 117.791699] usb 1-2.4: new full-speed USB device number 5 using tegra-xusb
[ 117.813803] usb 1-2.4: New USB device found, idVendor=0483, idProduct=5740
[ 117.813814] usb 1-2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 117.813821] usb 1-2.4: Product: STM32 Virtual ComPort
[ 117.813827] usb 1-2.4: Manufacturer: STMicroelectronics
[ 117.813832] usb 1-2.4: SerialNumber: 00000000001A

Without SSD use SD card
[ 260.787731] usb 1-2.3: new full-speed USB device number 10 using tegra-xusb
[ 260.813921] usb 1-2.3: New USB device found, idVendor=0483, idProduct=5740
[ 260.813944] usb 1-2.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 260.813953] usb 1-2.3: Product: STM32 Virtual ComPort
[ 260.813957] usb 1-2.3: Manufacturer: STMicroelectronics
[ 260.813961] usb 1-2.3: SerialNumber: 00000000001A
[ 260.816696] cdc_acm 1-2.3:1.0: ttyACM0: USB ACM device

Please help, thanks.

[BUG] NvRmMemInit failed - CUDA conflicts

Describe the issue
Please describe the issue

What version of L4T/JetPack
L4T/JetPack version: 4.6.3

Which Jetson
Jetson: TX2

To Reproduce
Steps to reproduce the behavior:
When I run this /usr/src/tensorrt/bin/trtexec --verbose --onnx=best.onnx --saveEngine=bestOnnx.engine --explicitBatch, some boots it works great but some boots it gave me an ****NvRmMemInit failed**** when trying to read CUDA device.
When it works, I get the normal graphical interface. When it doesn't, I get a flickering screen with no image.

Expected behavior
Run trtexec until the end.
Get graphical interface.

Additional context
I've tried to add NVIDIA services to the Before= fields but no success.

Jetpack is not installed on ssd.

While moving the boot to ssd before installing jetpack, an error message appears saying that the jetpack sdk installation is not possible...
image
In this case, is there a way to install jetpack from the ssd or to move the boot to emmc to install the sdk again or to reboot from the sd card?

[BUG] mount: special device /dev/nvme0n1p1 does not exist

Hello,

First, thank you to jetsonhacks for making these scripts available for everyone to use.

I ran into a little issue when running these scripts on my device and wanted to provide some details in case anyone else runs into the same issue.

While running the copy script I was getting an error that read: "mount: special device /dev/nvme0n1p1 does not exist"

After checking the default mount point of my SSD in the Disks utility I was able to confirm the actual mount point on my device is /dev/nvme0n1 not /dev/nvme0n1p1

To resolve the issue, I changed each instance of /dev/nvme0n1p1 to /dev/nvme0n1 in each of the following files:
copy-rootfs-ssd.sh
/data/setssdroot.service
/data/setssdroot.sh

After making those small edits the scripts run without issue and the system is now running from the NVMe SSD. Not sure why my Xavier is producing a device dev location different from the one used in the script, but I have provided details about my particular setup below:

What version of L4T/JetPack
L4T/JetPack version: L4T 32.4.3 [JetPack 4.4]

Which Jetson
Jetson: Xavier NX

Additional context
This Xavier NX was purchased on 10/5/2020
SSD is a 500GB Western Digital Blue SN550 - WDS500G2B0C

Return the root back to the SD card

Hello,

I used your scripts to change the root from SD card to SSD. Finally, it is not what I wanted. Is it possible to get back to the default set up?

Thanks!

[Not Bug] It works fine on Jetson Orin Nano Dev Kit

Describe the issue

This issue is just a report. The rootOnNVMe scripts work perfectly on the Jetson Orin Nano Dev Kit.
Thank you very much to make them. They are totally helpful.

sudo hdparm -t /dev/nvme0n1 /dev/mmcblk1
[sudo] password for youtalk: 

/dev/nvme0n1:
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 Timing buffered disk reads: 4088 MB in  3.00 seconds = 1362.06 MB/sec

/dev/mmcblk1:
 HDIO_DRIVE_CMD(identify) failed: Invalid argument
 Timing buffered disk reads: 202 MB in  3.02 seconds =  66.79 MB/sec

What version of L4T/JetPack
L4T/JetPack version: JetPack 5.1.1

Which Jetson
Jetson: Jetson Orin Nano Dev Kit

Not able to boot rootfs from SSD

Hello,
Thanks for sharing the srcipts, I followed the steps and ran the scripts to switch the rootfs to SSD successfully.
But after few days, the rootfs is loading from SSD, but stuck at nvidia logo, able to access only through terminal mode using clt+alt+f4. GUI is not responding. Disabling the service and removing the file setssdroot.conf from /etc and rebooting boots up from sd card and shows ssd unmounted. When I try to enable the service back again it doesn’t work. Can anyone help me resolve this issue. I faced this couple of days back, i had to format the SSD and again copy back the rootfs and scripts to run properly which is time consuming to install stuff in SSD again. This happened again now and I m just wondering if there is any other way to get back to SSD without formatting.

[BUG] Drivers for IMX477 camera are not showing up when root on SSD

Describe the issue
Drivers for IMX477 camera are not showing up when root on SSD
What version of L4T/JetPack
L4T/JetPack version: 32.4.4

Which Jetson
Jetson: Xavier NX

To Reproduce
Steps to reproduce the behavior:

  1. Boot from SSD (new system)
  2. Install the driver
  3. reboot
  4. in terminal type dmesg | grep imx477

Expected behavior
If you are booted from SD card and properly installed drivers from .deb, you will get the following:

Screenshot from 2020-12-03 18-36-38
THAT IS CORRECT !

Indicating, that drivers are working and trying to find camera.
However, if you are booted from SSD following steps from: jetsonhacks, after installation of the driver
NOTHING whill show up in: dmesg | grep imx477

How chould i make drivers working?

Additional context

We were using IMX477 camera with Xavier NX and when it is booted from SD card drivers working and the camera also works as intended.
However, when root booted on SSD and I install drivers there the same way, this time dmesg | grep imx477 shows no logs indicating that apparently that drivers are not working. I don't know if other people claiming their camera drivers are not working, dealing with the same issue, but is it possible to check this? I think it will help me and others a lot. We can provide any extra information relating my hardware and software configuration and any logs that are required.

Cold reboot loses SSD info[BUG]

Describe the issue
Please describe the issue
After copying over rootfs to SSD and then enabling booting from SSD, I reboot.
It works fine as long as I'm simply warm booting. That is I don't unplug the power.
When I unplug the power and then do a cold boot, the system doesn't see the SSD directories.
I have to do another warm boot (reboot) in order to see SSD directories.

What version of L4T/JetPack
L4T/JetPack version: 4.4
Which Jetson
Jetson: NXXavier
To Reproduce
Steps to reproduce the behavior:
For example, what command line did you run?

Setup the SSD.
Clone the rootOnNVMe
then copy rootfs
./copy-rootfs-ssd.sh
then enable booting from SSD
./setup-service.sh

Reboot to enable SSD.

Now you have access to SSD.

At this point, you can shutdown and unplug the power or simply just unplug the power.

When the power is reconnected, SSD directories are not seen.
You have to reboot again to see the SSD directories.
Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

[FEATURE REQUEST]

please explain how revert the changes, I had a no expected effect and I need revert it. would be nice with a script, and an explanation deeper of how upgrade or fix the system when get broken and do not boot by upgrade, and make the modules and everything get fine

[BUG] Fan becomes uncontrollable

Using the script, I successfully transferred EMMC to SSD on Xavier NX, but the fan seems to be uncontrollable. I installed jtop after the transfer.
Do you have any good suggestions?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.