Steps:

1. Write zero files to an archive. The archive file is created successfully.
2. Use mtar_open() to read the archive file.

Expected result: mtar_open() succeeds.
Actual result: mtar_open() fails with the "null record" error.

After writing N files into an archive (where N can possibly be zero), one would expect to read that archive and extract N files from it without a failure. Presently, when reading an archive, one has to check whether the tar file is empty (i.e., the file is 1KB of zeros, which is the end-of-archive entry). It would be nice if mtar_open() could handle this special case without failing.

I am getting checksum and other errors when trying to read a tar.xz.

Is it not supported ?

When attempting to archive an already tar.gz compressed file, it corrupts it and then I fail to extract.

Currently it seems you must know the size of the data going into the tar but in some cases where I'm porting IO to go into a tar, I don't know the file size and I'd prefer to avoid buffering.

Is is trivial to enhance this to keep updating the size as I write more data? I see some code to start writing null after size remaining is 0, perhaps I could simply modify that line to keep going?

Also would be cool if I could write multiple files at the same time (not multi-threaded though) by getting a descriptor when writing the file header. The API would then have you pass that descriptor to the write_data functions.

Appreciate the hard work :)

https://github.com/netpipe/LunaLibs/tree/master/mtar

been working on this works on windows fine still trying to get linux version working.

Hi,

Library is so nice. I want to integrate it in my application.
Do you have any small example to extract tar file in my current directory?

Thanks,
PG

The implementation implicitly uses unsigned, etc which is often compiled as 32-bit words. In this case offsets will silently wrap for tarballs longer than 4GB and cause errors.

The easiest fix would be to add a separate method (mtar_add_pos(unsigned pos, unsigned offset)) which checks that the position doesn't wrap silently while doing the addition. Support for a 64-bit implementation would be nice as well :)

I noticed that if I use mtar_finalize() the tar cannot be opened again for writing later. The files do not appear inside but the tar size grows.

If I skip calling finalize I can open the tar for writing in append mode and write more files to it without any problems. My tar clients seem to even be able to open them.

Is finalize truly needed? Should I just try to call it at the end of my process which opens and closes the tar multiple times to append it's data. Or as a compromise can I just "unfinalize" by removing the last two null records? Why doesn't the library do that?

Please add file with main() function and makefile like this

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..a3f6503
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,10 @@
+*.tmp
+*.o
+*.*~
+*.d
+*.dSYM
+*.swp
+
+test
+*.tar
+*.txt
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..ef8d87e
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,20 @@
+CC  = gcc
+BIN = test
+
+# Flags
+CFLAGS += -std=c99 -Wall -pedantic -pedantic-errors -O2
+
+SRC = $(BIN).c
+OBJ = $(SRC:.c=.o)
+
+$(BIN): ./src/microtar.o $(BIN).c
+	rm -f $(BIN) $(OBJS)
+	$(CC) $(SRC) -I./src $(CFLAGS) -o $(BIN) $(LIBS) ./src/microtar.o
+
+./src/microtar.o: ./src/microtar.c ./src/microtar.h
+	$(CC) $(CFLAGS) -c -o ./src/microtar.o ./src/microtar.c
+
+.PHONY: clean
+
+clean:
+	rm -f ./$(BIN) ./*.tar ./*.txt ./src/*.o
diff --git a/test.c b/test.c
new file mode 100644
index 0000000..0a906dc
--- /dev/null
+++ b/test.c
@@ -0,0 +1,29 @@
+#include <string.h>
+#include "microtar.h"
+
+int main(void)
+{
+
+mtar_t tar;
+/* utf-8 chars */
+const char *str1 = "Hello world, Witaj świecie";
+const char *str2 = "Goodbye world, żegnam";
+
+/* Open archive for writing */
+mtar_open(&tar, "test.tar", "w");
+
+/* Write strings to files `test1.txt` and `test2.txt` */
+/* no setup data */
+mtar_write_file_header(&tar, "test1.txt", strlen(str1));
+mtar_write_data(&tar, str1, strlen(str1));
+mtar_write_file_header(&tar, "test2.txt", strlen(str2));
+mtar_write_data(&tar, str2, strlen(str2));
+
+/* Finalize -- this needs to be the last thing done before closing */
+mtar_finalize(&tar);
+
+/* Close archive */
+mtar_close(&tar);
+
+return 0;
+}

Maybe that should be bold in the README.md. It's my bad for not reading it ... but maybe that can save some time to someone else.

I spent a long time debugging something, I had tar->pos set to something garbage instead of 0 while trying to read from a custom stream.

But thanks for that library, it works great for us !

I am wondering how easy it is to overwrite a single file within a tar without re-creating it?

If modify doesn't work, what about deleting the record and appending a new one at the end with new content?

Is possible to add function to set/read data creating file inside tar?

Any idea how I can bypass the character limit (99) of the file name ?
I have some extra long file names, and their names cannot be changed.

I don't know how good form this is, but this is useful when A) obviously when you have your tar file in mem, B) useful when you have a inflated (decompressed!) gzip buffer from e.g. zlib.

// mtar from mem buffer

static int mem_write(mtar_t *tar, const void *data, unsigned size)
{
	return MTAR_EWRITEFAIL;
}

static int mem_read(mtar_t *tar, void *data, unsigned size)
{
	unsigned char *buf = (unsigned char *)tar->stream;
	memcpy(data, buf + tar->pos, size);
	return MTAR_ESUCCESS;
}

static int mem_seek(mtar_t *tar, unsigned offset)
{
	return MTAR_ESUCCESS;
}

static int mem_close(mtar_t *tar)
{
	// Todo, Delete the data?
	return MTAR_ESUCCESS;
}

int mtar_open_mem(mtar_t *tar, void *data)
{
	int err;
	mtar_header_t h;

	// Init tar struct and functions
	memset(tar, 0, sizeof(*tar));
	tar->write = mem_write;
	tar->read = mem_read;
	tar->seek = mem_seek;
	tar->close = mem_close;

	tar->stream = data;

	err = mtar_read_header(tar, &h);
	if (err != MTAR_ESUCCESS)
	{
		mtar_close(tar);
		return err;
	}

	// Return ok
	return MTAR_ESUCCESS;
}
// --------

It is possible to cause stack-overflow while calling mtar_write_file_header and passing name of the file larger than 100.

Inside microtar.c strcpy is called which results in overwriting more data than it should.

==73490==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x00016b1a687c at pc 0x00010517be68 bp 0x00016b1a6770 sp 0x00016b1a5f20
WRITE of size 201 at 0x00016b1a687c thread T0
    #0 0x10517be64 in wrap_strcpy+0x4fc (libclang_rt.asan_osx_dynamic.dylib:arm64+0x4be64) (BuildId: 4947f3677e4435f39b5765e7dbc19bf732000000200000000100000000000b00)
    #1 0x104c5dab4 in mtar_write_file_header microtar.c:336
    #2 0x104c5a618 in LLVMFuzzerTestOneInput target.cc:19
    #3 0x104c76584 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) FuzzerLoop.cpp:617
    #4 0x104c75e78 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) FuzzerLoop.cpp:519
    #5 0x104c77550 in fuzzer::Fuzzer::MutateAndTestOne() FuzzerLoop.cpp:763
    #6 0x104c78394 in fuzzer::Fuzzer::Loop(std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>&) FuzzerLoop.cpp:908
    #7 0x104c6773c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) FuzzerDriver.cpp:912
    #8 0x104c94570 in main FuzzerMain.cpp:20
    #9 0x1a028ff24  (<unknown module>)
    #10 0xb47efffffffffffc  (<unknown module>)

Attached is a sample crash file.

crash.zip

Hi. Nice C thing. I wrote my mem_ variants from the file_ variants.

The tar->stream was only used in these four callbacks, so I thought this would work, but I get a -6 bad checksum.

static int mem_write(mtar_t *tar, const void *data, unsigned size)
{
	//unsigned res = fwrite(data, 1, size, (FILE *)tar->stream);
	//return (res == size) ? MTAR_ESUCCESS : MTAR_EWRITEFAIL;
	return MTAR_EWRITEFAIL;
}

static int mem_read(mtar_t *tar, void *data, unsigned size)
{
	unsigned char *buf = (unsigned char *)tar->stream;
	data = &buf[tar->pos];
	return MTAR_ESUCCESS;
}

static int mem_seek(mtar_t *tar, unsigned offset)
{
	return MTAR_ESUCCESS;
}

static int mem_close(mtar_t *tar)
{
	return MTAR_ESUCCESS;
}

int mtar_open_mem(mtar_t *tar, void *data)
{
	int err;
	mtar_header_t h;

	// Init tar struct and functions
	memset(tar, 0, sizeof(*tar));
	tar->write = mem_write;
	tar->read = mem_read;
	tar->seek = mem_seek;
	tar->close = mem_close;

	tar->stream = data;

	err = mtar_read_header(tar, &h);
	if (err != MTAR_ESUCCESS)
	{
		mtar_close(tar);
		return err;
	}

	// Return ok
	return MTAR_ESUCCESS;
}```