While extracting the full firmware of Oneplus 11 after extracting certain files it stops and throws error " input format not supported by decoder "

https://github.com/vm03/payload_dumper

Kinda like a GUI kind of setup in a sense, where you can manually extract and modify whatever partition you want.
Thanks in advance.

realme gt rmx2202 android 12

https://componentota-manual-cn.allawnfs.com/component-ota/21/10/17/8c89fffacdfe4f5b95bc576c10e0b15e.zip

@vm03

Could you explain with details and some example the use of different command line switch like

[--images IMAGES]

[--out OUT]

[--diff]

[--old OLD]

Thanks.

Hi,
This is just a minor issue. The dependencies only list protobuf from pip, but on a fresh Arch install, it also needs bsdiff4 from pip as well. Can the README be updated to contain bsdiff4 as well?

Stacktrace:
File "./payload_dumper.py", line 157, in
dump_part(part)
File "./payload_dumper.py", line 109, in dump_part
data = data_for_op(op,out_file,old_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "./payload_dumper.py", line 44, in data_for_op
data = dec.decompress(data)
^^^^^^^^^^^^^^^^^^^^
_lzma.LZMAError: Input format not supported by decoder

Rom:
Realme incremental rom and old rom:
Incremental: https://rbp01.realme.net/GT_Neo5_SE/RMX3700_11_A_OTA_0240_all_8oFIxq_10010111.zip
Old: https://rms11.realme.net/sw/RMX3700domestic_11_A.13_2023040323120179.zip

I am unable to extract the payload.bin from the firmware ColorOS 13 F.13 which is meant for instantnoodlep. There are no issues when extracting the same using other tools like https://github.com/ssut/payload-dumper-go or https://github.com/crazystylus/otadump

Processing my_stock partition...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Traceback (most recent call last):
  File "payload_dumper/payload_dumper.py", line 157, in <module>
    dump_part(part)
  File "payload_dumper/payload_dumper.py", line 109, in dump_part
    data = data_for_op(op,out_file,old_file)
  File "payload_dumper/payload_dumper.py", line 44, in data_for_op
    data = dec.decompress(data)
_lzma.LZMAError: Input format not supported by decoder

python3 payload_dumper.py --diff payload.bin
Processing devcfg partition....Done
Processing aop partition.....Traceback (most recent call last):
File "payload_dumper.py", line 157, in
dump_part(part)
File "payload_dumper.py", line 109, in dump_part
data = data_for_op(op,out_file,old_file)
File "payload_dumper.py", line 89, in data_for_op
out_file.write('\0' * ext.num_blocks*block_size)
TypeError: 'str' does not support the buffer interface

Traceback (most recent call last):
File "payload_dumper.py", line 163, in
dump_part(partition[0])
File "payload_dumper.py", line 109, in dump_part
data = data_for_op(op,out_file,old_file)
File "payload_dumper.py", line 44, in data_for_op
data = dec.decompress(data)
_lzma.LZMAError: Input format not supported by decoder

hyper rom:miui_CUPID_OS1.0.23.12.4.DEV_8bafbf5abd_14.0

error:Processing system_ext partition............................Traceback (most recent
call last):
File "payload_dumper.py", line 157, in
dump_part(part)
File "payload_dumper.py", line 109, in dump_part
data = data_for_op(op,out_file,old_file)
File "payload_dumper.py", line 44, in data_for_op
data = dec.decompress(data)
_lzma.LZMAError: Input format not supported by decoder

✘ ildar@ildar  ~/Загрузки  python3 payload_dumper.py payload.bin
Traceback (most recent call last):
File "/home/ildar/Загрузки/payload_dumper.py", line 15, in
import update_metadata_pb2 as um
File "/home/ildar/Загрузки/update_metadata_pb2.py", line 33, in
_descriptor.EnumValueDescriptor(
File "/home/ildar/.local/lib/python3.11/site-packages/google/protobuf/descriptor.py", line 914, in new
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
✘ ildar@ildar  ~/Загрузки 

https://source.android.com/devices/tech/ota/reduce_size?hl=ru#the_build_diff_tool
https://android.googlesource.com/platform/build/+/master/tools/releasetools/target_files_diff.py

the vbmeta.img suppose to be patched but payload_dumper handle it as a normal file, dumped vbmeta.img having magic bytes 'BSDF2'.

sample:
Full :https://gauss-componentotacostmanual-cn.allawnfs.com/remove-e39dd9079364ef1fbd57ae9092cf1192/component-ota/22/10/25/b0ff16a754294e96b78181901d008ca6.zip
Incremental :https://gauss-compotacostauto-cn.allawnfs.com/remove-65220760352fdd1e1cc0aacc346c9fce/component-ota/22/08/27/ba57c400420e44f8b53b7df7a9e10213.zip

reference:
https://android.googlesource.com/platform/external/bsdiff/+/77833b6477cd576fabd7211f2129918d7d372148

the vbmeta.img files dumped:
sample.zip

What is this project supposed to be licensed as?

https://choosealicense.com/

After manually deleting the "output" folder, an error occurred during operation.

STR:
Followed the instructions
Typed python payload_dumper.py payload.bin then enter
An error shows

  File "C:\Users\M\Downloads\dumper\payload_dumper.py", line 69
    <title>payload_dumper/payload_dumper.py at master · vm03/payload_dumper</title>
                                                      ^
SyntaxError: invalid character '·' (U+00B7)

Update ZIP: https://t.me/s/OnePlusOTA/89
System: Color OS 11.2 Android 11
Device: OnePlus 9 Pro

The unpacking process ended normally without any error, but the contents of the image file were damaged.

File "C:\Python312\payload_dumper.py", line 157, in
dump_part(part)
File "C:\Python312\payload_dumper.py", line 109, in dump_part
data = data_for_op(op,out_file,old_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python312\payload_dumper.py", line 44, in data_for_op
data = dec.decompress(data)
^^^^^^^^^^^^^^^^^^^^
_lzma.LZMAError: Input format not supported by decoder

I am extracting the payload.bin of my OnePlus Nord ce 3 Lite 5G but keep getting this error:

Traceback (most recent call last):
File "/storage/emulated/0/Android Rooting/payload_dumper.py", line 157, in
dump_part(part)
File "/storage/emulated/0/Android Rooting/payload_dumper.py", line 109, in dump_part
data = data_for_op(op,out_file,old_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/emulated/0/Android Rooting/payload_dumper.py", line 44, in data_for_op
data = dec.decompress(data)
^^^^^^^^^^^^^^^^^^^^
_lzma.LZMAError: Input format not supported by decoder

docker run error:

Processing boot partition................................................................................................Done
Processing vbmeta_system partition.Done
Processing my_stock partition.........................................................................................................Traceback (most recent call last):
  File "/app/payload_dumper.py", line 157, in <module>
    dump_part(part)
  File "/app/payload_dumper.py", line 109, in dump_part
    data = data_for_op(op,out_file,old_file)
  File "/app/payload_dumper.py", line 49, in data_for_op
    data = dec.decompress(data)
OSError: Invalid data stream

coloros 12:
https://component-ota-afs.coloros.com/component-ota/22/04/27/d7fc7eea4f5e4fc58c4792584c27e9c1.zip

When I'm try to extract payload.bin from new open beta 12 zip for Oneplus 9 pro, it fails. No matter ColorOS or Oxygen the errors always a same.
Processing my_heytap partition...........................................................................................................................................................................................................................................................................................................Traceback (most recent call last): File "D:\Oneplus9pro\payload_dumper-master\payload_dumper.py", line 157, in <module> dump_part(part) File "D:\Oneplus9pro\payload_dumper-master\payload_dumper.py", line 109, in dump_part data = data_for_op(op,out_file,old_file) File "D:\Oneplus9pro\payload_dumper-master\payload_dumper.py", line 44, in data_for_op data = dec.decompress(data) _lzma.LZMAError: Input format not supported by decoder

Can you check this. Thanks

On the installation step, I get this output:

[fabian@laptop extract]$ python -m pip install -r requirements.txt
Defaulting to user installation because normal site-packages is not writeable
Collecting protobuf<=3.20.1,>=3.19.3
  Downloading protobuf-3.20.1-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 13.2 MB/s eta 0:00:00
Requirement already satisfied: six>=1.16.0 in /usr/lib/python3.10/site-packages (from -r requirements.txt (line 2)) (1.16.0)
Collecting bsdiff4>=1.1.5
  Downloading bsdiff4-1.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (31 kB)
Installing collected packages: bsdiff4, protobuf
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
googleapis-common-protos 1.59.0 requires protobuf!=3.20.0,!=3.20.1,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 3.20.1 which is incompatible.
google-api-core 2.11.0 requires protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 3.20.1 which is incompatible.
Successfully installed bsdiff4-1.2.3 protobuf-3.20.1

Hi, @vm03! :)

Is there a solution to this problem? Only boot.img didn't succeed. I have Nokia 4.2

Thanks!

Log:

(.py) PS D:\Downloads\1> python payload_dumper.py --diff payload.bin
Processing aboot partition......Done
Processing dtbo partition......Done
Processing vbmeta partition.Done
Processing mdtp partition.................................................................................................................Done
Processing modem partition.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done
Processing sbl1 partition.............Done
Processing rpm partition...........Done
Processing tz partition..............Done
Processing devcfg partition...Done
Processing dsp partition.........................Done
Processing cmnlib partition...Done
Processing cmnlib64 partition...Done
Processing keymaster partition.......Done
Processing boot partitionUnsupported type = 9

All the below commands worked and installed fine
python.exe -m pip install --upgrade pip
python -m pip install protobuf
python -m pip install -r requirements.txt

When i run this
python.exe payload_dumper.py payload.bin

Traceback (most recent call last):
File "C:\Users\hamma\AppData\Local\Programs\Python\Python310\payload_dumper.py", line 15, in
import update_metadata_pb2 as um
File "C:\Users\hamma\AppData\Local\Programs\Python\Python310\update_metadata_pb2.py", line 8, in
from google.protobuf import reflection as _reflection
File "C:\Users\hamma\AppData\Local\Programs\Python\Python310\lib\site-packages\google\protobuf\reflection.py", line 58, in
from google.protobuf.internal import python_message as message_impl
File "C:\Users\hamma\AppData\Local\Programs\Python\Python310\lib\site-packages\google\protobuf\internal\python_message.py", line 62, in
from google.protobuf.internal import containers
File "C:\Users\hamma\AppData\Local\Programs\Python\Python310\lib\site-packages\google\protobuf\internal\containers.py", line 182, in
MutableMapping = collections.MutableMapping
AttributeError: module 'collections' has no attribute 'MutableMapping'

I have a functional requirement: I want to know which partitions the target payload.bin file has, so as to better extract the image file I want.

for example:
input：python payload_dumper.py --list payload.bin
output：
part_name size(bytes)
boot 100663296
......

Can you add the feature in latest push? 🌹

Android 13 introduced XOR compression and when patching the Incremental Update from it, it will give the output without errors but the files are wrong. It won't boot and the hashes are different.

Trying to dump a full OTA for OP10, Android 13 and I keep getting this error.

.............................Traceback (most recent call last):
File "D:\payload_dumper\payload_dumper.py", line 157, in
dump_part(part)
File "D:\payload_dumper\payload_dumper.py", line 109, in dump_part
data = data_for_op(op,out_file,old_file)
File "D:\payload_dumper\payload_dumper.py", line 44, in data_for_op
data = dec.decompress(data)
_lzma.LZMAError: Input format not supported by decoder
PS D:\payload_dumper>

FILE_HASH=ACc7D09Gt6rnyrI8/gYZFaZ/tCSDj8M3a3TwqaIHM5M=
FILE_SIZE=5113783109
METADATA_HASH=tnNDi+sAMw/P4TR3q0rROJIBETEhSfvHO6VMK6pjoMw=
METADATA_SIZE=199913
security_patch_vendor=2022-10-05
ota_target_version=NE2215_11.C.21_1210_202210231858
security_patch=2022-10-05
oplus_separate_soft=20847
oplus_rom_version=V13.0.0
oplus_update_engine_verify_disable=0

https://mega.nz/file/cTIj0b6R#ZO5aHMfTi9WPXmLxWqT2MuMZM8EkUkMEpZpGO8Z9RRU
or
https://drive.google.com/file/d/16mhmqRXs0neOsKg_a7ylRkfbHjsvKv0h/view?usp=share_link

https://ota-oversea-1256119282.file.myqcloud.com/SKW-H0_EEA/2019-09-19_2_skywalker_mp_os_ota4_20190904-2019-07-04_skywalker_mp_os_ota3_20190618_OTA3_EEA.20190920_154535.zip

Looks like it partially works atleast until it gets to the boot image and errors out...
Upon looking at the contents of the output folder using the --diff to merge incremental with dumped firmware it looks like the output filesizes are still smaller than expected on the other images as well..... I don't know if this is due to filesystem types of what? Running for a OnePlus Nord N20 5G. The GN2200 to be exact.

Here is an output from my fstab:
(I have removed the disclaimer notes between each section except the very first to save space)
Psyk0n4ut:
OP515AL1@android:/ $ su
root@android:/ # cat /vendor/etc/fstab

Copyright (c) 2019 - 2020, The Linux Foundation. All rights reserved.

Redistribution and use in source and binary forms, with or without

modification, are permitted provided that the following conditions are

met:

* Redistributions of source code must retain the above copyright

notice, this list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above

copyright notice, this list of conditions and the following

disclaimer in the documentation and/or other materials provided

with the distribution.

* Neither the name of The Linux Foundation nor the names of its

contributors may be used to endorse or promote products derived

from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED "AS IS" AND ANY EXPRESS OR IMPLIED

WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF

MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT

ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS

BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR

CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF

SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR

BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,

WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE

OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN

IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Android fstab file.

The filesystem that contains the filesystem checker binary (typically /system) cannot

specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

# <mnt_point> <mnt_flags and options> <fs_mgr_flags>
/dev/block/bootdevice/by-name/modem /vendor/firmware_mnt vfat ro,shortname=lower,uid=1000,gid=1000,dmask=227,fmask=337,context=u:object_r:firmware_file:s0 wait,slotselect

Copyright (c) 2019-2020 The Linux Foundation. All rights reserved.

Android fstab file.

The filesystem that contains the filesystem checker binary (typically /system) cannot

specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

# <mnt_point> <mnt_flags and options> <fs_mgr_flags>
system /system ext4 ro,barrier=1,discard wait,slotselect,logical,first_stage_mount
system_ext /system_ext ext4 ro,barrier=1,discard wait,slotselect,logical,first_stage_mount
product /product ext4 ro,barrier=1,discard wait,slotselect,logical,first_stage_mount
vendor /vendor ext4 ro,barrier=1,discard wait,slotselect,logical,first_stage_mount
odm /odm ext4 ro,barrier=1,discard wait,slotselect,logical,first_stage_mount
/dev/block/by-name/metadata /metadata ext4 noatime,nosuid,nodev,discard wait,formattable,first_stage_mount
/dev/block/bootdevice/by-name/persist /mnt/vendor/persist ext4 noatime,nosuid,nodev,barrier=1 wait#/dev/block/bootdevice/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,reserve_root=32768,resgid=1065,fsync_mode=nobarrier latemount,wait,check,formattable,quota,reservedsize=128M,checkpoint=fs
/dev/block/bootdevice/by-name/misc /misc emmc defaults defaults
/devices/platform/soc/4784000.sdhci/mmc_host* /storage/sdcard1 vfat nosuid,nodev wait,voldmanaged=sdcard1:auto,encryptable=footer
/devices/platform/soc/.ssusb/.dwc3/xhci-hcd..auto /storage/usbotg vfat nosuid,nodev wait,voldmanaged=usbotg:auto
/dev/block/bootdevice/by-name/modem /vendor/firmware_mnt vfat ro,shortname=lower,uid=1000,gid=1000,dmask=227,fmask=337,context=u:object_r:firmware_file:s0 wait,slotselect
/dev/block/bootdevice/by-name/dsp /vendor/dsp ext4 ro,nosuid,nodev,barrier=1 wait,slotselect
/dev/block/bootdevice/by-name/bluetooth /vendor/bt_firmware vfat ro,shortname=lower,uid=1002,gid=3002,dmask=227,fmask=337,context=u:object_r:bt_firmware_file:s0 wait,slotselect

Copyright (c) 2018 The Linux Foundation. All rights reserved.

Android fstab file.

The filesystem that contains the filesystem checker binary (typically /system) cannot

specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

# <mnt_point> <mnt_flags and options> <fs_mgr_flags>
system /system ext4 ro,barrier=1 wait,avb=vbmeta_system,logical,first_stage_mount
product /product ext4 ro,barrier=1 wait,avb=vbmeta_system,logical,first_stage_mount
vendor /vendor ext4 ro,barrier=1 wait,avb=vbmeta_vendor,logical,first_stage_mount
#ifdef VENDOR_EDIT
#[email protected], 2019/08/05, Modify for enable super partition
my_odm /odm ext4 ro,barrier=1 wait,logical,first_stage_mount,nofail
my_product /my_product ext4 ro,barrier=1 wait,logical,first_stage_mount,nofail
my_engineering /my_engineering ext4 ro,barrier=1 wait,logical,first_stage_mount,nofail
#other mount
#todo
my_company /my_company ext4 ro,barrier=1 wait,logical,first_stage_mount,nofail
my_carrier /my_carrier ext4 ro,barrier=1 wait,logical,first_stage_mount,nofail
my_region /my_region ext4 ro,barrier=1 wait,logical,first_stage_mount,nofail
my_heytap /my_heytap ext4 ro,barrier=1 wait,logical,first_stage_mount,nofail
my_stock /my_stock ext4 ro,barrier=1 wait,logical,first_stage_mount,nofail
my_preload /my_preload ext4 ro,barrier=1 wait,logical,first_stage_mount,nofail
#endif
/dev/block/by-name/metadata /metadata ext4 noatime,nosuid,nodev,discard wait,formattable,wrappedkey,first_stage_mount
#/dev/block/bootdevice/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,reserve_root=32768,resgid=1065,fsync_mode=nobarrier latemount,wait,resize,check,formattable,fileencryption=ice,wrappedkey,quota,reservedsize=128M,sysfs_path=/sys/devices/platform/soc/1d84000.ufshc,checkpoint=fs
/dev/block/bootdevice/by-name/cache /cache ext4 nosuid,noatime,nodev,barrier=1 wait/dev/block/bootdevice/by-name/persist /mnt/vendor/persist ext4 noatime,nosuid,nodev,barrier=1 wait/dev/block/bootdevice/by-name/boot /boot emmc defaults defaults
/dev/block/bootdevice/by-name/recovery /recovery emmc defaults defaults

/dev/block/bootdevice/by-name/misc /misc emmc defaults defaults
/devices/platform/soc/8804000.sdhci/mmc_host* /storage/sdcard1 vfat nosuid,nodev wait,voldmanaged=sdcard1:auto,encryptable=footer
/devices/platform/soc/1da4000.ufshc_card/host* /storage/sdcard1 vfat nosuid,nodev wait,voldmanaged=sdcard1:auto,encryptable=footer
/devices/platform/soc/.ssusb/.dwc3/xhci-hcd..auto /storage/usbotg vfat nosuid,nodev wait,voldmanaged=usbotg:auto
/dev/block/bootdevice/by-name/spunvm /mnt/vendor/spunvm vfat rw,shortname=lower,uid=1000,gid=1000,dmask=007,fmask=007,context=u:object_r:spunvm_file:s0 wait
/dev/block/bootdevice/by-name/modem /vendor/firmware_mnt vfat ro,shortname=lower,uid=1000,gid=1000,dmask=222,fmask=333,context=u:object_r:firmware_file:s0 wait
/dev/block/bootdevice/by-name/dsp /vendor/dsp ext4 ro,nosuid,nodev,barrier=1 wait/dev/block/bootdevice/by-name/bluetooth /vendor/bt_firmware vfat ro,shortname=lower,uid=1002,gid=3002,dmask=227,fmask=337,context=u:object_r:bt_firmware_file:s0 wait

#ifdef VENDOR_EDIT
#[email protected], 2017/12/29, Add for new reserve partition/dev/block/bootdevice/by-name/opporeserve2 /mnt/vendor/opporeserve ext4 nosuid,nodev,noatime,barrier=1 wait,check
#endif

Copyright (c) 2019-2020 The Linux Foundation. All rights reserved.

Android fstab file.

The filesystem that contains the filesystem checker binary (typically /system) cannot

specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

#[email protected], 2020/10/20, change the initial permission of vendor/firmware_mnt

# <mnt_point> <mnt_flags and options> <fs_mgr_flags>

system /system ext4 ro,barrier=1,discard wait,slotselect,avb=vbmeta_system,logical,first_stage_mount,avb_keys=/avb/q-gsi.avbpubkey:/avb/r-gsi.avbpubkey:/avb/s-gsi.avbpubkey
system /system erofs ro wait,slotselect,avb=vbmeta_system,logical,first_stage_mount,avb_keys=/avb/q-gsi.avbpubkey:/avb/r-gsi.avbpubkey:/avb/s-gsi.avbpubkey
system_ext /system_ext ext4 ro,barrier=1,discard wait,slotselect,avb=vbmeta_system,logical,first_stage_mount
system_ext /system_ext erofs ro wait,slotselect,avb=vbmeta_system,logical,first_stage_mount
product /product ext4 ro,barrier=1,discard wait,slotselect,avb=vbmeta_system,logical,first_stage_mount
product /product erofs ro wait,slotselect,avb=vbmeta_system,logical,first_stage_mount
vendor /vendor ext4 ro,barrier=1,discard wait,slotselect,avb=vbmeta_vendor,logical,first_stage_mount
vendor /vendor erofs ro wait,slotselect,avb=vbmeta_vendor,logical,first_stage_mount
odm /odm ext4 ro,barrier=1,discard wait,slotselect,logical,first_stage_mount,avb_keys=/vendor/etc/oplus_avb.pubkey
odm /odm erofs ro wait,slotselect,logical,first_stage_mount,avb_keys=/vendor/etc/oplus_avb.pubkey
/dev/block/by-name/metadata /metadata ext4 noatime,nosuid,nodev,discard wait,check,formattable,first_stage_mount
/dev/block/bootdevice/by-name/persist /mnt/vendor/persist ext4 noatime,nosuid,nodev,barrier=1 wait/dev/block/bootdevice/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,inlinecrypt,reserve_root=32768,resgid=1065,fsync_mode=nobarrier latemount,wait,resize,check,formattable,fileencryption=aes-256-xts:aes-256-cts:v2+inlinecrypt_optimized+wrappedkey_v0,keydirectory=/metadata/vold/metadata_encryption,metadata_encryption=aes-256-xts:wrappedkey_v0,quota,reservedsize=128M,checkpoint=fs
/dev/block/bootdevice/by-name/misc /misc emmc defaults defaults
#ifdef OPLUS_FEATURE_STORAGE_MOUNT
#[email protected], 2020/11/30, Add for REMOVE Adoptable
#/devices/platform/soc/4784000.sdhci/mmc_host* /storage/sdcard1 vfat nosuid,nodev wait,voldmanaged=sdcard1:auto,encryptable=footer
/devices/platform/soc/4784000.sdhci/mmc_host* /storage/sdcard1 vfat nosuid,nodev wait,voldmanaged=sdcard1:auto
#endif
/devices/platform/soc/.ssusb/.dwc3/xhci-hcd..auto /storage/usbotg vfat nosuid,nodev wait,voldmanaged=usbotg:auto
/dev/block/bootdevice/by-name/modem /vendor/firmware_mnt vfat ro,shortname=lower,uid=1000,gid=1000,dmask=222,fmask=333,context=u:object_r:firmware_file:s0 wait,slotselect

/dev/block/bootdevice/by-name/dsp /vendor/dsp ext4 ro,nosuid,nodev,barrier=1 wait,slotselect
/dev/block/bootdevice/by-name/bluetooth /vendor/bt_firmware vfat ro,shortname=lower,uid=1002,gid=3002,dmask=227,fmask=337,context=u:object_r:bt_firmware_file:s0 wait,slotselect
#ifdef OPLUS_FEATURE_STORAGE
#[email protected], 2020/12/24, Add for GKI
/dev/block/by-name/oplusreserve2 /mnt/vendor/oplusreserve ext4 nosuid,nodev,noatime,barrier=1 check,first_stage_mount,nofail
#[email protected], 2021/9/30 port the /mnt/oplus/op2 solution, to solve system process have no permission to /mnt/vendor
#use /mnt/oplus/op2 instead to fix the data/persist_log/oplusreserve occasionally mount fail problem
/dev/block/bootdevice/by-name/oplusreserve2 /mnt/oplus/op2 ext4 nosuid,nodev,noatime,barrier=1 check,first_stage_mount,nofail
#endif

Copyright (c) 2019-2020 The Linux Foundation. All rights reserved.

Android fstab file.

The filesystem that contains the filesystem checker binary (typically /system) cannot

specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

#[email protected], 2020/10/20, change the initial permission of vendor/firmware_mnt

# <mnt_point> <mnt_flags and options> <fs_mgr_flags>
system /system ext4 ro,barrier=1,discard wait,slotselect,avb=vbmeta_system,logical,first_stage_mount,avb_keys=/avb/q-gsi.avbpubkey:/avb/r-gsi.avbpubkey:/avb/s-gsi.avbpubkey
system /system erofs ro wait,slotselect,avb=vbmeta_system,logical,first_stage_mount,avb_keys=/avb/q-gsi.avbpubkey:/avb/r-gsi.avbpubkey:/avb/s-gsi.avbpubkey
system_ext /system_ext ext4 ro,barrier=1,discard wait,slotselect,avb=vbmeta_system,logical,first_stage_mount

system_ext /system_ext erofs ro wait,slotselect,avb=vbmeta_system,logical,first_stage_mount
product /product ext4 ro,barrier=1,discard wait,slotselect,avb=vbmeta_system,logical,first_stage_mount
product /product erofs ro wait,slotselect,avb=vbmeta_system,logical,first_stage_mount
vendor /vendor ext4 ro,barrier=1,discard wait,slotselect,avb,logical,first_stage_mount
vendor /vendor erofs ro wait,slotselect,avb,logical,first_stage_mount
odm /odm ext4 ro,barrier=1,discard wait,slotselect,avb,logical,first_stage_mount
odm /odm erofs ro wait,slotselect,avb,logical,first_stage_mount
/dev/block/by-name/metadata /metadata ext4 noatime,nosuid,nodev,discard wait,check,formattable,first_stage_mount
/dev/block/bootdevice/by-name/persist /mnt/vendor/persist ext4 noatime,nosuid,nodev,barrier=1 wait/dev/block/bootdevice/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,reserve_root=32768,resgid=1065,fsync_mode=nobarrier latemount,wait,check,formattable,encryptable=footer,quota,reservedsize=128M,checkpoint=fs
/dev/block/bootdevice/by-name/misc /misc emmc defaults defaults
/devices/platform/soc/4784000.sdhci/mmc_host* /storage/sdcard1 vfat nosuid,nodev wait,voldmanaged=sdcard1:auto,encryptable=footer
/devices/platform/soc/.ssusb/.dwc3/xhci-hcd..auto /storage/usbotg vfat nosuid,nodev wait,voldmanaged=usbotg:auto
/dev/block/bootdevice/by-name/modem /vendor/firmware_mnt vfat ro,shortname=lower,uid=1000,gid=1000,dmask=222,fmask=333,context=u:object_r:firmware_file:s0 wait,slotselect
/dev/block/bootdevice/by-name/dsp /vendor/dsp ext4 ro,nosuid,nodev,barrier=1 wait,slotselect
/dev/block/bootdevice/by-name/bluetooth /vendor/bt_firmware vfat ro,shortname=lower,uid=1002,gid=3002,dmask=227,fmask=337,context=u:object_r:bt_firmware_file:s0 wait,slotselect
root@android:/ #

from
cat /vendor/etc/*fstab

Hi. I'd like to say thanks 🎉 for this tool, very useful!

I have a suggestion: it shouldn't be too difficult to add a command-line option specifying which partition we want to extract - the others would just be ignored.

For example, I only wanted the boot.img to patch with Magisk (I suppose this is a common use-case). My SSD disk doesn't have much space for a zipped full OTA image, an unzipped payload.bin, plus all the partitions extracted...

I'd like to be able to write something like:

python payload_dumper.py payload.bin -p boot

which would only extract the small boot.img and nothing else. This would also make the process a lot faster.

Sorry for not providing a PR myself, but I never used Python, it would take some learning curve and wouldn't be trivial to me. Thanks in advance.

Hi, everyone so i want to extract the payload.bin and i got this:
python payload_dumper.py payload.bin
Traceback (most recent call last):
File "C:\Users\ngoan\AppData\Local\Programs\Python\Python310\payload_dumper.py", line 12, in
import update_metadata_pb2 as um
File "C:\Users\ngoan\AppData\Local\Programs\Python\Python310\update_metadata_pb2.py", line 8, in
from google.protobuf import reflection as _reflection
File "C:\Users\ngoan\AppData\Local\Programs\Python\Python310\lib\site-packages\google\protobuf\reflection.py", line 58, in
from google.protobuf.internal import python_message as message_impl
File "C:\Users\ngoan\AppData\Local\Programs\Python\Python310\lib\site-packages\google\protobuf\internal\python_message.py", line 62, in
from google.protobuf.internal import containers
File "C:\Users\ngoan\AppData\Local\Programs\Python\Python310\lib\site-packages\google\protobuf\internal\containers.py", line 182, in
MutableMapping = collections.MutableMapping
AttributeError: module 'collections' has no attribute 'MutableMapping'

If anyone has answer let me know. Thanks!

Hi, @vm03! :)

Is there a solution to this problem? vendor.img didn't succeed. I have skr-a0

Thanks!

Log:
Processing devcfg partition....Done
Processing aop partition.......Done
Processing xbl partition...............Done
Processing xbl_config partition...Done
Processing tz partition.........................Done
Processing hyp partition....Done
Processing dsp partition...............................................................Done
Processing keymaster partition.....Done
Processing cmnlib partition...Done
Processing cmnlib64 partition...Done
Processing qupfw partition...Done
Processing bluetooth partition..........................Done
Processing storsec partition...Done
Processing abl partition...Done
Processing modem partition.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done
Processing custom partition.....................................................................................................Done
Processing vendor partition.........................................................................................................................................................Unsupported type = 9

rebuild the payload.bin file ?

Hi, I tried to merge an incremental OTA with full using the -diff function. But I am getting an error:

narender@narender-MS-7816:~/Downloads/Payload Dumper/payload_dumper-master$ python3 payload_dumper.py --diff payload.bin
Processing xbl partition.Unsupported type = 10

The OTA is from an Android One device and should be complying with however payloads should be created. The full OTA did extract fine for me. Looking for a solution to merge the next incremental OTA to get a full output :)
Any help will be greatly appreciated.

python payload_dumper.py --diff payload.bin
Processing devcfg partition....Done
Processing aop partition.......Done
Processing xbl partition...............Done
Processing xbl_config partition...Done
Processing tz partition.........................Done
Processing hyp partition....Done
Processing dsp partition...............................................................Done
Processing keymaster partition.....Done
Processing cmnlib partition...Done
Processing cmnlib64 partition...Done
Processing qupfw partition...Done
Processing bluetooth partition..........................Done
Processing storsec partition...Done
Processing abl partition...Done
Processing modem partition..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done
Processing custom partition.....................................................................................................Done
Processing vendor partition.........................................................................................................................................................Unsupported type = 9

im getting this error message:
Traceback (most recent call last):
File "XXXXXXXXXXXXXX", line 7, in
import bsdiff4
ModuleNotFoundError: No module named 'bsdiff4'

First thank you for this script, it's been a life saver for flashing Android images. I'm trying to package this for Guix (See issue) but we need you to provide a license.

MacOS：Big sur 11.2.1

PS C:\Users*\Downloads\payload_dumper-master>
File "C:\Users*\Downloads\payload_dumper-master\payload_dumper.py", line 15, in
import update_metadata_pb2 as um
File "C:\Users*\Downloads\payload_dumper-master\update_metadata_pb2.py", line 33, in
_descriptor.EnumValueDescriptor(
File "C:\Users*\AppData\Local\Programs\Python\Python310\lib\site-packages\google\protobuf\descriptor.py", line 755, in new
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

There should be some kind of error when working with incremental OTA.
The resulting extracted boot.img has an "Unknown image format" when working with Magisk.

Upon attempting to run the script the user is alerted to the lack of bsdiff4. Installation of the module requires VS Build Tools, and installing the standalone version 14.0 a.k.a Build Tools 2015 does not allow pip to compile it with pip install bsdiff4. Would be worth adding to the README and adding instructions for compilation.

Run:python payload_dumper.py payload.bin
Appear:AssertionError: unsupported op
When me update payload_dumper version:
Processing abl partitionSOURCE_COPY supported only for differential OTA

Hi,
Can you add support for missing operations which were added after Android 8. Below is the list of operations which are not supported:

1. BROTLI_BSDIFF
2. PUFFDIFF
3. ZUCCHINI
4. LZ4DIFF_BSDIFF
5. LZ4DIFF_PUFFDIFF
6. DISCARD

Adding these will add support for incremental OTA in latest android releases like Android 12 and Android 13.

I tried flashing these weird images and my phone got stuck with the Linux logo. Thankfully I found a solution to fix it by changing my phone's slot from 'INVALID' to '_a' and reflashing stock images from the other tool called 'extract_android_ota_payload'

Traceback (most recent call last):
File "/home/fucker/Downloads/payload_dumper/payload_dumper.py", line 12, in
import update_metadata_pb2 as um
File "/home/fucker/Downloads/payload_dumper/update_metadata_pb2.py", line 8, in
from google.protobuf import reflection as _reflection
File "/home/fucker/.local/lib/python3.10/site-packages/google/protobuf/reflection.py", line 58, in
from google.protobuf.internal import python_message as message_impl
File "/home/fucker/.local/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 62, in
from google.protobuf.internal import containers
File "/home/fucker/.local/lib/python3.10/site-packages/google/protobuf/internal/containers.py", line 182, in
MutableMapping = collections.MutableMapping
AttributeError: module 'collections' has no attribute 'MutableMapping'