philipl / pifs Goto Github PK

In the section:
"So I've looked up my bytes in π, but how do I remember where they are?" - the "we all know" link seems to be broken,

alsoijw@debian:~/test$ mkdir 42
alsoijw@debian:~/test$ mkdir drive
alsoijw@debian:~/test$ πfs -o -mdd=42 drive/
πfs: Metadata directory must be specified with -o mdd=<directory>
alsoijw@debian:~/test$ πfs -o mdd=42 drive/
alsoijw@debian:~/test$ ls
ls: невозможно получить доступ к drive: Нет такого файла или каталога
42  drive
alsoijw@debian:~/test$ 

ls:it is impossible to access the drive: No such file or directory.
What I do wrong?

I use Ubuntu 9 x86
When I tried to make pifs I got :
πfs.c:258:14: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
I removed -Wall -Werror and build was successful.
When I ran : πfs -o mdd=meta/ fs/
all was ok.
Bu when I try work with fs folder I get "folder not found".

Is it work on x86?

I think that, if well played using integral calculus, we can use the filesystem to store important data and use the integration formula to provide the key to retrieve the data from the filesystem.

This way, we only need to find a way to represent such formula. Since the only thing that is necessary to save is the index and the formula, it can be prove to be an excellent way to conceal the data.

The rename syscall doesn't work in πfs. It always returns ENOENT, "No such file or directory".

To reproduce:

1. cd to a πfs mount point
2. Attempt to rename a file (e.g. mv foo bar, assuming foo exists)
3. Watch as it gives you an error

After mounting pifs (which works without an issue): pifs -o mdd=pifs mount

I can't do anything with the directory:

# cd mount
cd: no such file or directory: mount
# touch mount/test
touch: cannot touch ‘mount/test’: No such file or directory
# ls mount
/bin/ls: cannot access mount: No such file or directory
# mount
πfs on /home/.../pifs/mount type fuse.πfs (rw,nosuid,nodev,user=...)

Tried it as root as well, no effect.

Hi,

I have two more like Questions then Problems:

first of all is there the possibility to use multiple cores throughout the process of pifs?

second: Is there a way to check the progress of storing a file in pi?

thanks

I launch ./configure, and this gets spat out back to me.

./configure: line 1941: syntax error near unexpected token ;;' ./configure: line 1941: as_fn_error $? "unsafe absolute working directory name" "$LINENO" 5;;'

I have no idea what this means.

Yeah, that's right.

I cannot compile FUSE in Mac OS X and πfs doesn't use OSXFUSE.

during writing

for (index = 0; index < SHRT_MAX; index++) {
  if (get_byte(index) == *buf) {
    break;
  }
}
ret = write(info->fh, &index, sizeof index);

if you are not able to find index that makes "(get_byte(index) == *buf" to true,
index will be SHRT_MAX or 0...

So when you will decode it as

*buf = (char) get_byte(index);

you will receive wrong value.

Is it even proven that every finite sequence of digits exist somewhere in pi? If so, can you provide a link for that proof? Otherwise it feels like your concept is based on plausible reasoning at best...

https://tauday.com/tau-manifesto

i think pifs can then achieve 200% compression

http://www.qwantz.com/index.php?comic=2092

I tried to copy a small file into my pifs root directory and was informed, "No such file or directory."

Does this constitute a proof that pi is not normal?

I suppose it might be some problem with my installation:

FUSE library version: 2.9.0
fusermount version: 2.9.0
using FUSE kernel interface version 7.18

So, I was able to build everything, but I think it would be great to see some sample usage.

Perhaps something folks could even benchmark against?

Assume a moderately-sized file wanting to compress. Since there is a SMALL chance of the hexadecimal data as a singular part of pi. So why not multiple parts? Example is example:

Data: ABSODJ

Pi: AABBABFSODIJ

Data in Pi: aabbABdSODiJ

Metadata: Letters 5-6,8-11,13

I know the example is not using hexadecimal data, but I am too lazy.

Is it known (or, technically, hypothesized) what is the expected value of the index in pi of a given, random sequence of n bits? After all, if the logarithm of the index (i.e number of bits in index) grows linearly in the number of bits being stored, then storing the metadata is perhaps just as expensive as storing the data itself.

Euler's number (e) is also normal and in many cases more relevant. Historical precedent in normal-number filesystems has tended towards e. If a 'circular' number must be used, it should be tau.

I unziped pifs, then

cd pifs-master
./configure

bash:./configure: no such file or directory

Hey! So just say I find my data at index x.

Wouldn't the value of x be typically bigger than the data itself? And therefore cost just as much to store?

For example, say my data i wanted to store was "1234567890" and say that doesn't show up until index
1838448384248234818

Then storing that index would cost more than storing the data itself... I can see a case for some large #'s that happen to be stored at shorter index locations than the data itself, but how would this work otherwise?

instead of using just one index, use two indexes to move you along pi faster. the first index would be what number pi-prime, then the second being the index after that pi prime.

I built, installed, ran πfs -o mdd=<metadata directory> <mountpoint>, then what should I do?

As you probably already know, in three days GDPR compliance must be implemented for all storage solutions that process data of European citizens.

In your README I noticed the following line:

π holds every file that could possibly exist!

This sounds extremely alarming as it looks like in three days use of this technology will be considered illegal. Please advice on how I would go about removing my personal data from π.

While parsing the contents of PiFS I was shocked to find that it contained a large amount of objectionable and pornographic content. There are a set of indexes (which I won't reproduce here for legal reasons) point to files which I believe may cause the user to face serious liability if PiFS is discovered on their system.

In addition, there are indexes which contain numerous copyrighted materials, as well as keys to break various DRM schemes. I believe that the use of PiFS may constitute a DMCA violation.

fuse not found (for Debian/Ubuntu)

check if you have installed libfuse-dev package NOT fuse

errors during compilation

πfs.c: In function ‘pifs_opendir’:
πfs.c:258:14: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
   info->fh = (uint64_t) dir;

Edit ./makefile and ./src/makefile , search -Wall , and delete all behind "=".

Everyone knows Euler's constant and Tau are better constants that scale better to cloud infrastructure. They also both contain every finite sequence of digits of Pi in them anyway.

It would be cool if we could use these clearly superior constants instead of Pi, which I believe is deprecated and should not be used for anything in the age of web scale and the cloud. Due to recent breakthroughs like the internet of things - it would be cool if you let users choose which mathematical constant to use instead of Pi.

By the way, one easy way to calculate Pi based on e is to take a circle with radius e and then compare the ratio between its diameter and circumference so you could introduce an abstraction layer.

I can't install it on OSX.

Firstly, why doesn't the README file say that I have to run autogen.sh before running make?

Secondly, after running autogen.sh, I receive the following error:

configure: error: Package requirements (fuse >= 2.8) were not met:
Requested 'fuse >= 2.8' but version of fuse is 2.7.3

I did install FUSE for OS X version 2.8.0 beta from here. Then I also restarted the computer, to no avail.

Why is it telling me that I have fuse 2.7.3 installed? Why isn't the README mentioning that Fuse is required for this?

Given that most modern computers are capable of utilizing their brand new and overpowered graphics cards, it would be cool if pifs could use that power.
Maybe there should be some compilation option in pifs - compile regular, compile with OpenCL support, compile with CUDA support, etc.?
That would be a pretty cool part of this great experiment!

So after seeing in issue 8 that I must install autogen and fuse from sourceforge, I did so.

But I still got...

./autogen.sh: 9: ./autogen.sh: autoreconf: not found

In the end of day, I got this fixed by simply...

sudo apt-get install autoconf

...and reinstalling fuse from the buntu repos as the SF one is old. I knew something was fishy considering Linux Mint 15 already has these two packages anyway.

I suggest updating INSTALL and README with something like this, seeing as how the fix is so simple (it is 2 years old...).

Thank you for reading.
Jack.

It would be great If you provide us with enough of documentation,Architecture and Implementation details.

I compiled πfs and tried to mount a directory with it:

$ ls 
bin  mdd  root
$ ./bin/πfs root -o mdd=mdd
$ cd root
-bash: cd: root: No such file or directory
$ ls -lh | grep root
d????????? ? ?    ?       ?              ? root

What's wrong?

The README states "They said 100% compression was impossible? You're looking at it!"

There's a fundamental flaw in this statement:
Compression % = 100 * ( 1 - ( Compressed Filesize / Uncompressed Filesize ) )

So for instance we compress 100 MB of data into a 20 MB archive, then we have a compression of:
100 * ( 1 - ( 20 / 100 ) ) = 100 * ( 1 - 0.2 ) = 100 * 0.8 = 80%

In order for us to have a 100% compression, we'd have to compress our files down to 0 bytes.

We still require massive amounts of storage to index the digits of pi and we still need storage space for the software that is supposed to calculate the digits of pi in order to retrieve our information.

So it's probably best to state that this method is almost 100% compression instead.

"They said 100% compression was impossible? You're looking at it!"
instead of compressing files, I'm ending up with files about 53846 times bigger than there size

Now when I try to remove test, it says that the file doesn't exist but it is still there.

My os is:
Linux antergos 4.4.5-1 x86_64

I want to start by saying that I am not a mathematician (in fact I am very bad with mathematics), so this may be really dumb:

It occurs to me that if it were possible to "fuzz" the matching a little bit, then store some error correction data along with the size and index, would that dramatically reduce the time necessary to calculate the initial position for writes? For example, find the first occurrence that is really close to the file's data (maybe within 97% or so), then calculate a binary diff/patch that would be necessary to "fix" the data after retrieval. There are a lot of ways to do checksums and data recovery (usually for combating corruption), and we have the advantage of knowing ahead of time what bits will be corrupt!

It is possible this would take just as long as (or longer than) finding an exact match (and I am not sure how much the error correction data stored would reduce the compression ratio), but I was curious whether it would be possible.

Any thoughts from people who aren't horribly lost in the world of mathematics?

Hello,
I have generated one file like this:

echo 80 | xxd -r -p > junk

and then

pifs -o mdd=/some_dir/ mount_ponit_dir/
cp junk mount_ponit_dir/

and then terminal freezes.
It seems that the desired pattern in Pi take too time.

Would chunking the file lead to faster results? As long as size of the start:end in pi is smaller than the chunk of the file you are looking for it would be a win and then be stored as start1:end1,start2:end2...........startN:endN. Of course once you have this long set of pairings you could run that through the same process to shrink it further.

It can be proven that the size of index will be greater than the data. How can a file system store data with much larger than it need?

Installation steps as follows:

git clone [email protected]:philipl/pifs.git
cd pifs; ./autogen.sh

Errors during autogen:

Any ideas?

Upon doing a code review, it seemed apparent to me that a lookup table of size USHORT_MAX could significantly improve the performance of read() and write().

The README says that

One of the properties that π is conjectured to have is that it is normal, which is to say that its digits are all distributed evenly, with the implication that it is a disjunctive sequence, meaning that all possible finite sequences of digits will be present somewhere in it.

My organisation requires all software to be formally proved to always give the correct output. The fact that it is conjectured is not enough. Please add a unit test that proves the normality of pi.

PiFS sets a dangerous precedant ! thousands of PiFS Users have ALL my files!
THIS IS A PRIVACY NIGHTMARE AND AN OUTRAGE !

πfs -o mdd=pie pie

I accidentally use the same dir for mounting and mdd, it was tragic

Git is ignoring Makefile and configure which are important files that are needed to compile.

I tried to save a 46.3 kb sized file. After about 45 minutes the process terminated, but the file on the pifs was 0 bytes. Smaller files have been saved correctly.

Btw: nice idea, but there's still a lot of work to do ;)

I would suggest distributing the pifs filesystem source itself as a pifs disk image.

I'm just pointing out that this would be very space-conserving and easily upgradeable since the newest version and the future releases of pifs are already contained within pi.