Crypto Compression Concept Worth Big Money - I Did It!

[B(asic)Miner]

You guys, check THIS out!  I can't believe this, and it's on Wikipedia, too!

https://en.wikipedia.org/wiki/Jan_Sloot


Read this!  It sounds like he thought of the same thing I'm thinking of ....  holy cow!

[Kazimir]

Kazimir:  YOU SIR, ARE GOOD.  I had a great laugh over that one.  You got us both dead to rights.  And I knew it was a math trick,

Well, instead of a "math trick" I'd rather call it "sheer logic".

Anyway. Do we now agree that your idea is not possible? If not, then do we at least agree that this:

any 5MB file we try to compress (or encode) must be reduced to a unique 4KB crypto key

must be true in order for your idea to be possible?

[spndr7]

Sho is this Mexxi joker?  Is he someone from this site who is just messing around with me, or is this guy for real?

Mexxi
Member Join Date
Feb 2010

Posts
73

I think its pure coincidence

[Kazimir]

Just curious:  what about e?

Not known (yet), just like pi.

[Kazimir]

You guys, check THIS out!  I can't believe this, and it's on Wikipedia, too!

https://en.wikipedia.org/wiki/Jan_Sloot

Read this!  It sounds like he thought of the same thing I'm thinking of ....  holy cow!

Jan Sloot was proven to be a fraud long ago, yet there are still "believers" who prefer to ignore common sense and defy logic.

[B(asic)Miner]

It goes against my own interest to publish this here, but I will do so because of the hard work of the people who have followed me so far.  The man sounds a lot like me.  If he hadn't of died in 1999, I would have I was his reincarnated self trying to do this all over again.  It is strange he died on 9/11 1999, the year many think the government went online with some kind of technology which could see into the future which told them that 9/11 was going to happen if they wanted to take control of the world (which they do, so it did) ...  I'm not saying I believe that, but what if his idea was possible, and it's all been covered up?  That would mean by trying to recreate it (although I had no knowledge of the man before today) I would be asking for trouble.  *I* might have a heart attack if I try to create this ... if you get me.  Maybe some corporation (a data storage corporation) doesn't want this secret getting out which would bankrupt them?   Jeesh, I am becoming as paradoid as this man.
--------------

Translated from UT News:

In 1999, Jan Sloot, a common television repair man, discovered a revolutionary coding system that would make hard disks, CD-ROMs, and other data stores superfluous. All movies ever made would fit on one CD-ROM. Sloot attempted to sell his invention to Philips, but he found that Philips' engineers were not interested.

Roel Pieper, at that time a member of the board of Philips, thought the invention was valuable, and decided to join the inventor and his companions. Pieper and Sloot looked for investors all around the world. Pieper himself invested millions in the project. All investors of The Fifth Force, Sloot's company, hoped to become billionaires.

But the paranoid inventor died on September 11, 1999, one day before the invention would be specified in detail in a contract. All his notes, his prototype, and his source code were lost forever.

Translated from GIDS:

The Sloot Digital Coding System (SDCS) would shake the world: a new alphabet for digital storage that didn't use binary code, but a much more efficient method. The principle behind SDCS seems simple. As a text consists of a limited number of characters, a movie consists of a limited amount of colours and sounds. All those basic data were stored in five algorithms in five memory stores. For movies, each algorithm would have a maximum length of 74 Mb. That's 370 Mb in total: the invention's engine. To start the engine, only a proper key was needed. For every page of a book, for every image in a movie, Sloot calculated a unique code. The concatenation of these codes would again result in a unique code. The final code, the key, would be one kilobyte in length, regardless of the length of the movie or the size of the book.

Roel Pieper, professor in Computer Science, commented on people questioning the possibility of such a high compression (translated from UT News):

"It is not about compression. Everyone is mistaken about that. The principle can be compared with a concept as Adobe-postscript, where sender and receiver know what kind of data recipes can be transferred, without the data itself actually being sent."

So we're back to the alien and his stick. Basically, Roel Pieper explains Sloot's 'invention' as the movies already residing on the computers of both parties, but in a deconstructed form, and a key being transferred that specifies exactly how reconstruction must take place.

In principle, this is possible. But the crux is, how long the key needs to be, and how big the data store of deconstructed forms must be. Sloot claims that a key of one kilobyte, and a data store of less than 400 Mb, would suffice. This sounds unbelievable, and it is, in fact, mathematically impossible. There are many different arguments. I prefer the simplest one:

A key of a limited length (whether it is a kilobyte or a terabyte) can only store a limited number of codes, and therefore can only distinguish a finite number of movies. However, the actual number of possible movies is infinite. For, suppose it were finite; in that case there would be a movie that is the longest. By just adding one extra image to the movie, I would have created a longer movie, which I didn't have before. Ergo, the number of possible movies is infinite. Ergo, any key of limited length cannot distinguish every possible movie.

The SDCS is only possible if keys are allowed to become infinite, or the data store is allowed to become infinite (if the data store already contains all movies ever made, a key consisting of a number can be used to select the movie you want to see -- however, in that case it is impossible to have keys for movies that have not been made yet at the time the data store was constructed). This would, of course, make the idea useless.

The whole problem of Pieper's way of thinking (and of every other person who believes that Sloot actually could compress movies with a factor of two million) is that he believes that the key does not need to contain every detail of a movie. Unfortunately, it does.

In the SDCS, it is claimed that no movies are stored, only basic building blocks of movies, such as colours and sounds. So, when a number is presented to the SDCS, it uses the number to fetch colours and sounds, and constructs a movie out of them. Any movie. No two different movies can have the same number, otherwise they would be the same movie. Every possible movie gets its own unique number. Therefore, I should be able to generate any possible movie by loading some unique number in the SDCS.

Think of it: by placing the right number in the SDCS, I can not only get Orson Welles' Citizen Kane. I can get Citizen Kane in colour! Or Citizen Kane backwards! Or Citizen Kane where the credits misspell the name of Everett Sloane, who plays mr. Bernstein, as Everett Sloot! Or Citizen Kane in Swahili! Or Citizen Kane where 'Rosebud' is a teddy bear! Or Citizen Kane with a cameo of myself! Or the cartoon version of Citizen Kane! Or Citizen Kane where Charles Foster Kane wins the election! Or the gay version of Citizen Kane! Or Citizen Kane where Charles Foster Kane is replaced by Jar Jar Binks!

How many movies are possible that are variations on Citizen Kane? More than fit in a number of one kilobyte, I can tell you. More than there are atoms in the universe. An infinite number, actually.

In the SDCS, the details of every possible movie have to be somewhere. They are either in the number, or in the data store. Since the data store is the same for each movie, the details can't be in the data store. Therefore, everything that differs between movies, must be represented in some way in the numbers. Which, for infinite variations of movies, is impossible with finite numbers.

Would it be possible to store, say, all possible movies up to three hours in length, with a limited resolution and a limited sound quality, in a series of finite numbers? Yes, of course that is possible, since in the digital world, there is no difference between data and numbers, thus data of a limited length is equal to a finite number.

Even though Sloot claimed that one kilobyte was enough to store any movie regardless of its length, the question is warranted whether a system as defined by the SDCS is possible for movies of a limited (but realistic) length. The answer is no. One kilobyte is 1024 bytes, and a byte can take 256 different values. With a code of one kilobyte, to represent any movie up to 90 minutes in length, on average a byte must be able to distinguish between all possible sequences of about 5 seconds. Can a sequence of 5 seconds of any movie at all be identified with a number between 0 and 255? Of course not.

The concept of the SDCS, as described, is equivalent to a universal Turing machine. A universal Turing machine (which is actually quite easy to simulate in a computer), can emulate any possible computer program (including a program that shows Citizen Kane which ends with a pie fight) just by feeding it a number. The problem is that even for trivial programs the numbers quickly get enormous, so that it is impossible to filter the rare useful programs from the universe of chaos consisting of all numbers. Therefore, it isn't viable business to sell people a universal Turing machine as a solution to all their business problems.

Evidently, Jan Sloot was a crank. He showed all the signs attributed to cranks. He was paranoid. He felt himself an expert in a field he had no higher education in. He worked alone. He spent decades on one problem any mathematician could have told him was impossible to solve in the first place. He got angry at people who questioned him. He felt misunderstood. He had delusions of grandeur.

I think Jan Sloot really believed he had found a way to compress movies to one kilobyte. Probably he had imagined something as the alien's stick, something that would be theoretically possible in an alternate world, with infinitely divisible particles, but impossible in our reality. He probably lacked the mathematical insight to see that his 'invention' was a farce. In his prototype, he faked his invention, which is why he refused to let anyone near it, and answered only in mystical vagueness to questions. He probably imagined that, with enough money, he could get his invention to work for real. He probably did not feel himself a charlatan or a crook, but an honest businessman, who needed a starting capital to create the world's greatest invention, make some people incredibly rich, and win a Nobel prize.

What truly amazes me is that a man as Roel Pieper, who is a professor of Computer Science no less, could fall for his story, to the point where he invested a huge amount of capital. If his role in this story is really as reported in the media, his credibility as a computer scientist has been seriously tarnished. In my opinion, the University of Twente, with which Pieper is associated, should at least perform an internal investigation, to assess whether Pieper's position can be maintained.

Incidentally, if anyone is interested, I have devised an invention where I can store any movie ever created on a small stick. I need some money to develop this concept further. If you have a couple of million to spare, give them to me. I'll be glad to build a prototype. We're gonna be rich and famous!

Article Written September 28, 2004

[Jace]

Sweet Jesus, not that sh*t again

I knew it.. In a thread like this, the name "Jan Sloot" was bound to show up sooner or later

[BurtW]

As someone with vast experience in mpeg video this quote made my day:

The Sloot Digital Coding System (SDCS) would shake the world: a new alphabet for digital storage that didn't use binary code, but a much more efficient method. The principle behind SDCS seems simple. As a text consists of a limited number of characters, a movie consists of a limited amount of colours and sounds.

I never heard of "Jan Sloot" before today.  Thanks!

[B(asic)Miner]

Because of this article, I can sort of understand a lot better than before, why you all have problems with my theory, or I guess why *I* should have a problem with my own theory.  Is there something about this discovery that is calling people like me out of the wood works to try and solve it?  Does the universe want this to be created at this time to solve a fundamental need in culture?  Our world is rapidly running out of room to store the vast data we are capturing every moment.  I want to believe this man had a true idea, but that someone didn't want him to bring that one forward at that time.  Does that make me a conspiracist?  The missing card that contained his program, his death, the date of his death, all seem suspect.  I guess you guys have all heard of this guy before, so I'm sorry to bring it up again, but wow.  I feel sort of deflated again after that.

I am truly trying to achieve what I said I am trying to.  I am not a crackpot or a prankster.  To those of you who have already done some work, such as programming small files to help calculate number is Pi, would you PLEASE UPLOAD those to me somehow, so I can use them myself?  The work you have done shouldn't be lost if I decide to keep plugging along on this.

The different between me and Jan Sloot is that I've shared the full theory with you, so now it's become open source, I guess. So maybe there won't be anyone coming to give me a heart attack, and maybe if I die anyway, then someone can still see how I was thinking and try to go it on there own.  But I doubt I'll get rich now, or even be remembered.  I've lost the chance to have done something astounding because now anyone who reads this forum entry will also know how similar I am to Jan Sloot and just go back to the "he's a wacko" argument.

[BurtW]

Wow, that post reminds me of President Reagans favorite joke:

Worried that their son was too optimistic, the parents of a little boy took him to a psychiatrist. Trying to dampen the boy’s spirits, the psychiatrist showed him into a room piled high with nothing but horse manure. Yet instead of displaying distaste, the little boy clambered to the top of the pile, dropped to all fours, and began digging.

“What do you think you’re doing?” the psychiatrist asked.

“With all this manure,” the little boy replied, beaming, “there must be a pony in here somewhere.”

[B(asic)Miner]

Let me just add this story before I call it a night:

When I was in the Army, I had a programming friend who was good with Amigas.  He created a videogame that we never had a chance to publish because someone stole his computer before it was completed, just like days before the completion.  We worked on it for months.  When creating the animation system for the game and the tracking system for the tiles and how they would move and animate, he got the clever idea of using text blocks to represent different aspects of the state of the tile.

It looked like this for one tile:   FDCBG  ....  five letters.  The first letter was the x, then y, then initial state, then animation frame on currently, and then several other factors.

To get the tiles to animate, he would append every tile on the board into one long superstring like this:

FDCCGFDCBGFDCBGFDCBGFDDBGFDCBGFDCBGFDCBGFDCAGFDCBGFDCAAFDCBGFDCBGFDCBGFDCCBG  and then use CHR$ RIGHT$ LEFT$  MID$ functions to grab out what was needed for processing.

Now that one string told the game where every file was, what it was doing, its animation cycle, everything needed to display a huge number of tiles on screen while simultaneously animating them at different rates, different start points, end points, everything.  This one code represented everything happening on the screen at one time.  It would grow and shrink as certain tiles got deleted, moved, replaced, etc...   I felt this was genius.

One small code could tell the game everything it needed about each individual tile.   Talk about data compression, talk about creating a system that could replace what would take other people hundreds of lines of code for each tile and be less efficient ...  it amazed me.  I think that's ultimately led me to find such a thing as this in the first place.  That was back in 1992, so I've been thinking about it a long long long time.

I just want one of two things:   1) for my quest to be over by being proven it can't work (in a public way, not in some secret board room) ... or 2) finding it can work and getting the reward for not giving up on it.

Thanks guys.

[Mota]

Your idea works. Just not like you want it to. It does not work on large files (5TB) and it will not work on small files (under a few mb). Other than that it could be a reasonalbe approach to data encoding. But again, look at the links I provided you, you will find a lot of stuff there, including efficiency marks for several compressng programs up to 98% compression for certain types of files.

[foggyb]

B(asic)Miner, have you considered the possibility that maybe Jan Snoot (or whatever) really WAS just a crackpot? An ambitious crackpot, but a crackpot nonetheless?

[spndr7]

Highly Compressed data (39 KB)



Uncompressed data (39 KB)

1 pixel represents one bit
black pixel for '1'
white pixel for '0'

Entropy can be seen in the first image.

[murraypaul]

"I'm sorry to report, but your key has resulted in 12 collisions.... wait.... accessing the Basekey information ....  unique file found from key."

Now, although there were 12 collisions (where other files started and ended at the exact same place, had the same number of 1's in their files structure, and otherwise fit all the criteria, 11 of them had at least 1 bit different in the initial 64K Bit Key included in the 4K file, so the program was successfully able to weed out the other 11.   But let's say instead of that, we get this message:

"From unique key sampling, there are still 2 collisions, what would you like to do?"    You access:  "save both to disk."  Now you have two files on your desktop.  You try the first one in your videoplayer and it doesn't work.  You try the 2nd, and your movie is playing right before your eyes.  You take about an hour to try and see what the 2nd file is ... you try it as PDF, Doc file, Zip file, audio, everything, finally you discover it's a 3DS Max scene file from some guy in Florida who does 3D work for some studio there.  Amazing, it's exactly the same size as your file down to the last bit, but it's an actual working file that someone else made.  Now you have to wonder:  who made the file?  That man, or Nature?  Nature came first.  And it's numbers don't change.  Maybe the man discovered the art he is making through accessing Nature?  Creepy.

But perhaps this is how we solve the overlapping argument once and for all:  we let the software detect collisions and give the user options about how to work with them ....  

A single 5 byte file: "Hello", results is many many more than two million collisions.
There will only ever be more collisions the longer the file is.
A 100MB movie file would result in more collisions than could be saved to a hard drive.

You are going to ignore this, by saying that your theory magically only works on large files not small ones, which conviently means that we can't run a test program to demonstrate you are wrong.
But you are wrong.
If two 5 byte combinations give the same hash, then any pair of files which are exactly the same, except that they each start with a different one of those 5 byte combinations, will also give the same hash.
Making the file bigger just stops us demonstrating that you are wrong, it doesn't make you right.

[artbct]

I think it's good for a person to have a problem like this in the background of your brain; if for no other reason than it can be energizing and help you to become a better problem solver.

I myself am stuck on a stabilization system that is probably impossible, but it's a great brain exercise, and not letting go of it helps me to solve problems in the physical world often better than before I got stuck on the idea.

I had previously not heard of Jan Sloot and being that I'm a sucker for a good conspiracy theory and this makes the conversation even more fun for me:)

BTW thanks for supplying those links Moto:

And for the OP:
Please read into the stuff you want to do a little further:
http://mattmahoney.net/dc/dce.html#Section_11
http://www.maximumcompression.com/index.html

[FloridaBear]

Please read about Information Theory and Coding Theory.

The ability to compress any given set of data is based on the entropy of the data. Random noise is essentially uncompressible, regardless of what type of compression you try to apply.

[murraypaul]

I downloaded 1 billion digits of Pi from here: http://stuff.mit.edu/afs/sipb/contrib/pi/
I downloaded the complete works of Gilbert and Sullivan (text format) from here: http://www.gutenberg.org/files/808/

The ~1.3MB file needs just over 100 million digits of Pi to encode.

After byte 1295987, pi position is 103683939
After byte 1295988, pi position is 103684011
After byte 1295989, pi position is 103684099
After byte 1295990, pi position is 103684138

That is almost exactly 80 digits of Pi required for each 1 byte of input files.
(Which is exactly what you would expect.)

So a 100MB file would require 8.4 billion digits of Pi.
A 50GB file would require 4.2 trillion digits of Pi.

The current record for calculating digits of Pi is 10 trillion, and that took 371 days to do, so I think we agree real-time calculation is out of the question?
4.2 trillion digits of Pi require 4.2 trillion bytes to store uncompressed. That is a 4 Terabyte drive just to store Pi to be able to perform your 'compression'.
(Which doesn't work anyway, because it is not reversible.)

[murraypaul]

I just want one of two things:   1) for my quest to be over by being proven it can't work (in a public way, not in some secret board room) ... or 2) finding it can work and getting the reward for not giving up on it.

We have proven it can't work, repeatedly.
You just don't believe us.

[B(asic)Miner]

I downloaded 1 billion digits of Pi from here: http://stuff.mit.edu/afs/sipb/contrib/pi/
I downloaded the complete works of Gilbert and Sullivan (text format) from here: http://www.gutenberg.org/files/808/

The ~1.3MB file needs just over 100 million digits of Pi to encode.

After byte 1295987, pi position is 103683939
After byte 1295988, pi position is 103684011
After byte 1295989, pi position is 103684099
After byte 1295990, pi position is 103684138

That is almost exactly 80 digits of Pi required for each 1 byte of input files.
(Which is exactly what you would expect.)

So a 100MB file would require 8.4 billion digits of Pi.
A 50GB file would require 4.2 trillion digits of Pi.

Thanks for your hard work, and this information is invaluable to me.

However, you are still trying to assume I'm going to try and do the entire file in one attempt through Pi, when I already said I was going to limit the size to 500 MB.  But now I see that 100 MB requiring 8.4 billion digits of Pi is far too much, so I must lower my expectations for each chunk.  I will now say each chunk should be 25 MB, which is 1.1 Billion digits of Pi.

We would end up with something like this:

[OPENFILE]
[FileSplitSizeInKB=25600_&_LastChunkSizeInKB=9500]
[Filesize=409600_&_Name="400MBofStuffForWhomever.zip"]
[MChunks=8_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[2,w, xxxxxx, y, zzzz]
[3,w, xxxxxx, y, zzzz]
[4,w, xxxxxx, y, zzzz]
[5,w, xxxxxx, y, zzzz]
[6,w, xxxxxx, y, zzzz]
[7,w, xxxxxx, y, zzzz]
[8,w, xxxxxx, y, zzzz]
[CLOSEFILE]

This chunking method would have an additional obvious benefit of making the MetaData more unique.  But I still have to find a solution as to the problem of having 6 bits being 1s and 10 bits being 0s inside of 16 bit pairs and what that means toward my theory first.  

I do also need to work out a way of creating more uniqueness too, I agree.