Crypto Compression Concept Worth Big Money - I Did It!

[Kazimir]

<.< I did not state that this is the case with every irrational number now, did I?

Not trying to get off topic here, but you said "Since pi is an INFINITE number all of the data in the world is stored somewhere in there." and this argument is simply incorrect. Being "infinite" (what does that mean by the way.. not having a finite decimal representation? would that make 1/3 "infinite" as well?) by no means implies that all of the data in the world is stored somewhere in there. As my counter-example demonstrates.

As for Pi, is it actually not known yet if all data in the world is stored somewhere in there, or not.

[B(asic)Miner]

B(asic)Miner, please answer this:

You claim for any input file, you can create a 4K 'crypto key', from which you can re-create the original file.

Suppose we take two different files that are each 5K large. Let's call them A and B. Through some smart process (whose technical details are irrelevant for now) we calculate their corresponding crypto keys which are 4K each, i.e. two pieces of 4K data. Let's call these P and Q. Now, if it's possible (by means of some other smart process that may or may not involve generating Pi or whatever) to reconstruct A from P, and B from Q, do you agree that if A and B are different, then P and Q must be different as well?

(If not, i.e. if P = Q, then reconstructing the original file from P (or Q, which is the same) will either result in A, thus we can never re-create B, or it results in B, thus we can never re-create A)

I hope you are not setting me up for some math trick just to shame me.  I will try my best to answer this, but I feel I am walking into a trap.  I will give you the benefit of the doubt.  First, although I've taken extensive IQ tests online, and all my results hover at around 158 (some 145, one said 163), that aside, I barely graduated high school due to my body's reaction to stress.  I was a slow learner, a deep thinker yes, but slow because I had disabilities due to my strong emotional nature, whereby the slightest provocation would create intense emotions which would shut down my brain and cause me to be unable to think.  Being in school was always provocational, there were always bullies, and I was weak, a toe-headed small-bodied wimp, begging to be picked on.  My own home life was filled with step-brothers who saw me the same way.  I had no chance to learn to enjoy school or my own home life either.  Whenever I try to think today (as a legacy from those terrible early years), I get nervous, unless I'm alone, which causes me to slow down and become almost moronic in public, saying the most stupid things out loud.  So please please don't sandbag me with some math trick.  I'll try my best, but if you're intent was to destroy me in some way, please reconsider going through with it.  In the off chance that you truly want me to understand something important here, I will try to answer you.

First, my theory won't support files that small, its pointless to even bother with files that small, so I'm changing your input to 5 MBs, okay?

Each 5MB file A & B gets reduced to 4K, becoming P & Q.  As far as trying to now use another method to recapture the data, I must say sorry, there would be no other viable method, since it was sent through Pi, it must come out through Pi.  We could encode the same A & B files thru 2/3 or another irrational number but we would get different Metadata (Index).  

I agree that if A & B are different, P & Q must be different as well, yes.  

If P=Q, then what you have are two identical files, which can't be P & Q anymore, they would only be P.  Q would just be another copy of P.  That would mean A and B were both identical to start with, and in the beginning of the math problem, you said they were different, meaning P cannot equal Q, therefore this problem becomes irrelevant.

Please don't get angry, this is how I see it.  Maybe I don't understand.  I apologize.
  

[ZephramC]

Another thing is, you still can't convince me that just because it's possible to have 2^N files TO encode, that there are that many unique files.  For all we know, our research into this could reveal a fundamental characteristic of Nature to only allow organized patterns to assemble in random data at a given ratio, like the Golden Means (8/7 is it)?  Just trying to solve this problem itself could lead to a breakthrough.  What if there are only (2^N)/10 file in existence of each size, and they already happen to be written in Nature?    

This argument is hard to beat :-]]].
Of course there ARE NOT (exist somewhere on disk, in memory, written as book) 256^1024 of 1 kB files. But theoretically there could be.
What you are basically saying is that you algorithm will work for all "reasonable meaningful files". There is indeed very low number of them. But this is out of the scope of mathemeatics, because you can not define what exactly is "a file that means/will mean somethinh reasonable.

But all 2^N files can be created by simple computer program. (For some very high N bellow some limit dictated by current technology.). There is no known fundamental characteristic of Nature which prohibits some combination of bytes in file!
Or do you believe that some of 256^1048576  1MB files are physically impossible to create (save, copy, print)

[Kazimir]

I hope you are not setting me up for some math trick just to shame me.  I will try my best to answer this, but I feel I am walking into a trap.  I will give you the benefit of the doubt.

No worries. Just trying to either prove you wrong, or be proven wrong myself, through a logical argument. Not trying to trick anyone, merely hoping it will bring clarity on the subject.

First, my theory won't support files that small, its pointless to even bother with files that small, so I'm changing your input to 5 MBs, okay?

Yep, that's fine.

Each 5MB file A & B gets reduced to 4K, becoming P & Q.  As far as trying to now use another method to recapture the data, I must say sorry, there would be no other viable method, since it was sent through Pi, it must come out through Pi.  We could encode the same A & B files thru 2/3 or another irrational number but we would get different Metadata (Index). 

I agree that if A & B are different, P & Q must be different as well, yes.

OK, we agree so far. Now, in order for this approach to work for 5MB files in general, any 5MB file we try to compress (or encode) must be reduced to a unique 4KB crypto key, right? (otherwise, we'd have a conflicting situation where two different 5MB files A and B would reduce to the same 4KB crypto key, which we just agreed was not the case)

By the way, when I said "by means of some other smart process" I just meant the decompressing or reconstruction part does something else than the compression or encoding part, which was kinda obvious (in fact they complement eachother), but never mind, it's not important for now.

[B(asic)Miner]

<.< I did not state that this is the case with every irrational number now, did I?

Not trying to get off topic here, but you said "Since pi is an INFINITE number all of the data in the world is stored somewhere in there." and this argument is simply incorrect. Being "infinite" (what does that mean by the way.. not having a finite decimal representation? would that make 1/3 "infinite" as well?) by no means implies that all of the data in the world is stored somewhere in there. As my counter-example demonstrates.

As for Pi, is it actually not known yet if all data in the world is stored somewhere in there, or not.

I agree with that partially.  It doesn't mean all data is stored in Pi in easy to read linear fashion.  But all of life is made up of patterns, and for sure in every software program there are redundant expressions used over and over again by coders who want to maximize their time by copying old chunks and familiar lines of code that once made them famous in their fields.  Fundamentally, all of programming code, when rendered down to the most basic computing language is just binary, just 0s and 1s.  I am not saying people who think they can use strings and then hunt through Pi to locate those strings and turn it into Metadata are on the right trail.  I think they are not.  It's too slow, for starters.

My idea produces a thumbprint based on a file that is already created.  Trying to enter just any random Metadata (what I call Crypto Key) set into my software would only produce noise unless you just happened to stumble across a set that was already in there.  I think that's the real reason, if only on a subconscious level, people are against my idea, is that this notion is terrifying.  Imagine putting in a random Crypto Key and spitting out a video of 2016 showing Russia invading the U.S. just like in Red Dawn the movie, only for real?  That would violate time and space.  It seems a bit terrifying, but yes, I've thought about that possibility, and once when I thought about giving up on my theory, I almost decided to just write a short story about this instead.  But the desire to actually create something that does truly work is way too strong yet.

What is important is that whatever data you do put into it, produces a Metadata set that only works with my theory.  But if you change the base Index structure (not Pi but another number) you get different Meta Data.  Perhaps the thing to look at here is that Pi cannot contain everything, perhaps Pi only contains 1/42nd of everything.  Perhaps my theory needs to have multiple irrational numbers.

What if there are 42 irrational numbers and each of them contains part of the whole of the collected data and what if we need to process certain files through different irrational number schemes in order to capture the totality of the uniqueness needed to break the counting argument?  Perhaps one data set is too small to encapsulate everything, but all irrational numbers combined, it would?  The problem then would be in trying to figure out what files need to go to what irrational numbers for their uniqueness to be preserved?

That sounds like math I think no one could achieve.  But then again ... I won't say it's impossible because I don't believe in that myself.  Everything is possible somehow, if we only knew.

[Kazimir]

I agree with that partially.

Sorry, what's there not to agree with? Either Pi's decimal representation contains any possible sequence of digits, or it doesn't (i.e. there exists at least one specific sequence of digits that does not occur anywhere in Pi's decimals). Which of these two statements is true, is not known yet. It has neither been proven, nor falsified.

[ZephramC]

I hope you are not setting me up for some math trick just to shame me.  I will try my best to answer this, but I feel I am walking into a trap.  I will give you the benefit of the doubt.

No worries. Just trying to either prove you wrong, or be proven wrong myself, through a logical argument. Not trying to trick anyone, merely hoping it will bring clarity on the subject.

First, my theory won't support files that small, its pointless to even bother with files that small, so I'm changing your input to 5 MBs, okay?

Yep, that's fine.

Each 5MB file A & B gets reduced to 4K, becoming P & Q.  As far as trying to now use another method to recapture the data, I must say sorry, there would be no other viable method, since it was sent through Pi, it must come out through Pi.  We could encode the same A & B files thru 2/3 or another irrational number but we would get different Metadata (Index). 

I agree that if A & B are different, P & Q must be different as well, yes.

OK, we agree so far. Now, in order for this approach to work for 5MB files in general, any 5MB file we try to compress (or encode) must be reduced to a unique 4KB crypto key, right? (otherwise, we'd have a conflicting situation where two different 5MB files A and B would reduce to the same 4KB crypto key, which we just agreed was not the case)

By the way, when I said "by means of some other smart process" I just meant the decompressing or reconstruction part does something else than the compression or encoding part, which was kinda obvious (in fact they complement eachother), but never mind, it's not important for now.

I think that I can pedict your next argument and the next argument of B(asic)Miner :-]
Kazimr: Imagine long column of all 5MB files on the left and long column of all 4KB crypto keys on the right. Then connect with line which file on the left gets compressed to which crypto key on the right. Soon you will run out of crypto keys and some lines will point to the same crypto key...

B(asic)Miner: Most of the files on the left will never need to be compressed as they will never be produced in this Nature. (Because Nature allows only some reasonable/natural patterns.)

[ZephramC]

Everything is possible somehow, if we only knew.

Do you believe that it is "somehow" possible to find the largest prime number. Such as no larger prime number will ever be found?

[B(asic)Miner]

I agree that if A & B are different, P & Q must be different as well, yes.

OK, we agree so far. Now, in order for this approach to work for 5MB files in general, any 5MB file we try to compress (or encode) must be reduced to a unique 4KB crypto key, right? (otherwise, we'd have a conflicting situation where two different 5MB files A and B would reduce to the same 4KB crypto key, which we just agreed was not the case)

Yes, I agree.  But only partially.  If we included a function of the software that did the following, it might still work:

"I'm sorry to report, but your key has resulted in 12 collisions.... wait.... accessing the Basekey information ....  unique file found from key."

Now, although there were 12 collisions (where other files started and ended at the exact same place, had the same number of 1's in their files structure, and otherwise fit all the criteria, 11 of them had at least 1 bit different in the initial 64K Bit Key included in the 4K file, so the program was successfully able to weed out the other 11.   But let's say instead of that, we get this message:

"From unique key sampling, there are still 2 collisions, what would you like to do?"    You access:  "save both to disk."  Now you have two files on your desktop.  You try the first one in your videoplayer and it doesn't work.  You try the 2nd, and your movie is playing right before your eyes.  You take about an hour to try and see what the 2nd file is ... you try it as PDF, Doc file, Zip file, audio, everything, finally you discover it's a 3DS Max scene file from some guy in Florida who does 3D work for some studio there.  Amazing, it's exactly the same size as your file down to the last bit, but it's an actual working file that someone else made.  Now you have to wonder:  who made the file?  That man, or Nature?  Nature came first.  And it's numbers don't change.  Maybe the man discovered the art he is making through accessing Nature?  Creepy.

But perhaps this is how we solve the overlapping argument once and for all:  we let the software detect collisions and give the user options about how to work with them .... 

WAIT:  OMG!  This is the answer!    We use the collision detection DURING ENCODING AS WELL.  This would dramatically increase encoding time, because you'd have to decode the file to see if there were collisions before allowing it be used as your save point.  You could try a list of irrational numbers until you found the one that gave you the unique ending spot!  It's so awesome!   (But yes, slow!  Very very slow!) 

Or just allow for the Collision Detection after the fact and work with some tricks to try and pair down the collisions found.

[Mota]

<.< I did not state that this is the case with every irrational number now, did I?

Not trying to get off topic here, but you said "Since pi is an INFINITE number all of the data in the world is stored somewhere in there." and this argument is simply incorrect. Being "infinite" (what does that mean by the way.. not having a finite decimal representation? would that make 1/3 "infinite" as well?) by no means implies that all of the data in the world is stored somewhere in there. As my counter-example demonstrates.

As for Pi, is it actually not known yet if all data in the world is stored somewhere in there, or not.

Infinite means per definition not ending. 1/3 is not infinite since 0,33333333333 (periodic) is finite written as 1/3. Per definition 0,99999 (period) is defined as 1. Periodic numbers are seen as finite numbers in general. Pi is seen as infinite since it is an irrational constant. Which means there is an infinite amount of numbers contained within it -> ergo my statement that every data is included there. But you are right, I can't prove that. But you can't prove the opposite

[Mota]

And for the OP:
Please read into the stuff you want to do a little further:
http://mattmahoney.net/dc/dce.html#Section_11
http://www.maximumcompression.com/index.html

[B(asic)Miner]

I hope you are not setting me up for some math trick just to shame me.  I will try my best to answer this, but I feel I am walking into a trap.  I will give you the benefit of the doubt.

No worries. Just trying to either prove you wrong, or be proven wrong myself, through a logical argument. Not trying to trick anyone, merely hoping it will bring clarity on the subject.

First, my theory won't support files that small, its pointless to even bother with files that small, so I'm changing your input to 5 MBs, okay?

Yep, that's fine.

Each 5MB file A & B gets reduced to 4K, becoming P & Q.  As far as trying to now use another method to recapture the data, I must say sorry, there would be no other viable method, since it was sent through Pi, it must come out through Pi.  We could encode the same A & B files thru 2/3 or another irrational number but we would get different Metadata (Index). 

I agree that if A & B are different, P & Q must be different as well, yes.

OK, we agree so far. Now, in order for this approach to work for 5MB files in general, any 5MB file we try to compress (or encode) must be reduced to a unique 4KB crypto key, right? (otherwise, we'd have a conflicting situation where two different 5MB files A and B would reduce to the same 4KB crypto key, which we just agreed was not the case)

By the way, when I said "by means of some other smart process" I just meant the decompressing or reconstruction part does something else than the compression or encoding part, which was kinda obvious (in fact they complement eachother), but never mind, it's not important for now.

I think that I can pedict your next argument and the next argument of B(asic)Miner :-]
Kazimr: Imagine long column of all 5MB files on the left and long column of all 4KB crypto keys on the right. Then connect with line which file on the left gets compressed to which crypto key on the right. Soon you will run out of crypto keys and some lines will point to the same crypto key...

B(asic)Miner: Most of the files on the left will never need to be compressed as they will never be produced in this Nature. (Because Nature allows only some reasonable/natural patterns.)

Kazimir:  YOU SIR, ARE GOOD.  I had a great laugh over that one.  You got us both dead to rights.  And I knew it was a math trick, but at least you took the humorus approach to it rather than the "AHA! GOTCHA SUCKA" approach.  Too funny!   That's exactly what I would have said, too.  Or tried so say,  You said it better.  That is also funny.

[BurtW]

I almost decided to just write a short story about this instead.  

That is a pretty good idea there.

What is important is that whatever data you do put into it, produces a Metadata set that only works with my theory.  But if you change the base Index structure (not Pi but another number) you get different Meta Data.  Perhaps the thing to look at here is that Pi cannot contain everything, perhaps Pi only contains 1/42nd of everything.  Perhaps my theory needs to have multiple irrational numbers.

42?  Are you a Hitchhikers Guide fan?

What if there are 42 irrational numbers and each of them contains part of the whole of the collected data and what if we need to process certain files through different irrational number schemes in order to capture the totality of the uniqueness needed to break the counting argument?  Perhaps one data set is too small to encapsulate everything, but all irrational numbers combined, it would?  The problem then would be in trying to figure out what files need to go to what irrational numbers for their uniqueness to be preserved?

hmmm...

[kslavik]

I agree that if A & B are different, P & Q must be different as well, yes.

OK, we agree so far. Now, in order for this approach to work for 5MB files in general, any 5MB file we try to compress (or encode) must be reduced to a unique 4KB crypto key, right? (otherwise, we'd have a conflicting situation where two different 5MB files A and B would reduce to the same 4KB crypto key, which we just agreed was not the case)

Yes, I agree.  But only partially.  If we included a function of the software that did the following, it might still work:

"I'm sorry to report, but your key has resulted in 12 collisions.... wait.... accessing the Basekey information ....  unique file found from key."

Now, although there were 12 collisions (where other files started and ended at the exact same place, had the same number of 1's in their files structure, and otherwise fit all the criteria, 11 of them had at least 1 bit different in the initial 64K Bit Key included in the 4K file, so the program was successfully able to weed out the other 11.   But let's say instead of that, we get this message:

"From unique key sampling, there are still 2 collisions, what would you like to do?"    You access:  "save both to disk."  Now you have two files on your desktop.  You try the first one in your videoplayer and it doesn't work.  You try the 2nd, and your movie is playing right before your eyes.  You take about an hour to try and see what the 2nd file is ... you try it as PDF, Doc file, Zip file, audio, everything, finally you discover it's a 3DS Max scene file from some guy in Florida who does 3D work for some studio there.  Amazing, it's exactly the same size as your file down to the last bit, but it's an actual working file that someone else made.  Now you have to wonder:  who made the file?  That man, or Nature?  Nature came first.  And it's numbers don't change.  Maybe the man discovered the art he is making through accessing Nature?  Creepy.

But perhaps this is how we solve the overlapping argument once and for all.

The problem with your "Partial Agree" is that there will be not 12 collisions, there will be infinite number of collisions.

[spndr7]

Is this a real coincidence ?

Compression with Pi


A forum topic created in encode.ru yesterday.

[BurtW]

Yes, I agree.  But only partially.  If we included a function of the software that did the following, it might still work:

"I'm sorry to report, but your key has resulted in 12 collisions.... wait.... accessing the Basekey information ....  unique file found from key."

But perhaps this is how we solve the overlapping argument once and for all.

Sure, only for very large files the message may be more like:

"I'm sorry to report, but your key has resulted in 12,234,123 collisions.... wait.... accessing the Basekey information ....  now there are only 10,234,123 possible files."

[B(asic)Miner]

Is this a real coincidence ?

Compression with Pi


A forum topic created in encode.ru yesterday.

It's not a coincidence ... but it makes me extremely nervous.  Because I already posted my same idea over there and now someone else is trying to claim he had the idea, when I posted on the very same forum the same day I posted here.

Check it out here:    http://encode.ru/threads/1789-100-Lossless-Compression-Theory-Please-Join-Me?highlight=SmallestFilePoss

I B(asic)Miner am called SmallestFilePoss on that forum.  You can check my Skype and Yahoo information on that site for cross reference that I'm telling the truth.

Sho is this Mexxi joker?  Is he someone from this site who is just messing around with me, or is this guy for real?

[BurtW]

Mexxi's responses on that forum seem very reasonable.

[Kazimir]

Infinite means per definition not ending. 1/3 is not infinite since 0,33333333333 (periodic) is finite written as 1/3. Per definition 0,99999 (period) is defined as 1. Periodic numbers are seen as finite numbers in general. Pi is seen as infinite since it is an irrational constant. Which means there is an infinite amount of numbers contained within it -> ergo my statement that every data is included there. But you are right, I can't prove that. But you can't prove the opposite

For Pi, no, I can't. But I have given an example of another irrational constant, which also has an infinite amount of numbers contained within it, yet some numbers are not in there.

Hence the argument "x is infinite, thus every date is included there" is not true in general.

In fact, if a number if infinite, anything is possible:

Number	Infinite?	Every data included?
1/3	no	no
0.121221222122221etc	yes	no
Chaitin's constant	yes	yes
√2	yes	unknown!
Pi	yes	unknown!

[BurtW]

Just curious:  what about e?