Bitcoin Forum

Hello Everyone,

The Achievement:
I have made an amazing discovery and spent a year trying to talk everyone I know into helping me with my solution for compressing 99.8% of all data out of a file, leaving only a basic crypto key containing the thread of how to re-create the entire file from scratch.  Its a truly elegant solution, a truly stunning discovery worth millions of dollars possibly.  But I'm just a teacher and don't have a fancy title or famous or powerful friends.  I am what society considers "a nobody" ... just your average guy.  But I like to solve puzzles, and I believed in myself, and I think that counts for more than being a nobody overall.  I achieved my goal, but now I can't seem to get it recognized or developed.  Imagine what I know I can now do:   with a software version of my concept for data compression, you could take your 12-gig Blueray version of Lord of the Rings 3, compress it down to under 600-1200KB (the size of one jpeg image) and email it to your friend in moments.  What's more, imagine that this file, because it no longer contains any data at all, just crypto data, is now 100% unrecognizable, scannable, or readable from any third party inbetween who might intercept your file.  It's basically 100% unreadable data, 99.8% compressed data.  This is what I know I have achieved.

Back Story:
Starting in 2009, I began to have day dreams about creating something merely by thinking about it.  I was looking around for ideas about something that could be created merely by thought, not a product of any kind, because I don't have those skill sets.  After a while, because of my love of hackers, at first I thought I would create a game of some kind.  I created an awesome game idea, one which would take all of those useless brain games and put them into an architecture that would make them important for solving a larger over-arching goal of hacking a mainframe.  In the course of creating the game (and yes, this game idea would itself be worth millions if you knew everything it entailed), I came across a far more important idea, the idea of being able to compress real-life data in all computer-based files down to absolutely (just short of) nothing.

In 2011, I worked on it the whole year in my head while at my real job, my teaching job.  I figured out a way to do it even then, but it was so tedious in trying to explain it to my one hacking buddy who was smart enough to be able to create the program for me that he just gave up listening to me.  He wanted to help me, but I had no programming experience and wasn't able to explain the idea.  I wasn't about to give up though, so I went back and thought about how to resolve my theory down to a distilled set of rules.  I went from like 20 rules ... down to four.  This took another year.  I actually (whether you believe me or not) had a dream on December 21st, 2012 (the day the world was supposed to end) that showed me how to fix my problem with the theory I currently was working under.  It was so easy, it blew my mind.  Its so elegant, I believe my solution for compressing 99.8% of all data out of a file is something that deserves to win the Nobel Prize.  It might be remembered in similar fashion to Einstein or Tesla.

What Do I Want by Telling All of You This?:
I am hoping to find smart individuals who believe me about this, who have the means to start a company with me, a joint venture, of which I myself have no starting capital of any kind.  I have always been poor, I come from a poor family, and have no powerful friends or allies.  I'm currently trying to get started in Bitcoin mining to make a little money to finally begin forming a savings.  My job is very very poor, I work in a foreign country.  I'm sort of trapped here by the fact that 1) its the best job I can get with no college degree, and 2) America's unemployement is up to at least 20%, far above what is being reported by the corporate-owned media.  It's too risky to return to America and try to find work.  I must try to make something of this idea, because it promises to be my one and only shot at escaping my situation in life.

I am willing to share equally all incomes derived from the creation of my idea if you assume the risk.  I'm hoping to find an investor who wants to take a gamble.  I'm certain my idea about how to do compression will work, I have hand tested the idea numerous times, and am able to reproduce data merely from the crypto-code my theory generates.  That one code can recreate an entire file, every last bit, out of thin air.  It's not magic.  The Bible says all things are possible for them that believe.  Okay, I'm paraphrasing.  But yes, I believed I could do it, and now I have it, and I don't know what to DO WITH IT.  

I don't even have the money to patent the concept, and my family won't help me because I'm a thinker who has been thinking of ideas for 25 years and they just don't believe a word I say about my ideas anymore.  I'm desperate to prove everyone wrong who ever doubted me.  This is my chance.  Will someone please take a chance and help me start a company?

Where Will This Lead?
1) Reducing Internet Traffic Worldwide.  Speeding up transfers of all datum.  Imagine Sony PS4 letting you download an entire game online in mere moments, and then decompressing it to the hard-drive (might take a while to do that part) .... But the internet wouldn't suffer, it would get cleaned up in a massive way.
2) Government Uses
3) Every file sent over the internet is encrypted and totally unreadable.
4) Every file in existence can be catalogued by its resulting crypto-code, making storing a backup of all data online fit on one hard drive.  The entire contents of the internet and every computer in the world could fit on one small hard drive in encoded form.
.... and just for fun:
5) One day, teleportation projects could benefit from being able to send a small packet of data that contains the entire DNA re-sequencing of the person being teleported.  Star Trek could become a reality, because you wouldn't have to send billions of Terrabytes of Data through space, just send the small crypto-code, way less energy required.  Almost none, in fact.

FINALE:
There was a TV show episode about this ("Person of Interest") starring that guy from Lost ("Ben" the island leader) where they talked about someone who had done what I am claiming I have done.  It was fictional.  I was laughing when the TV show aired because I had already basically achieved the idea myself, but it had yet to become efficient enough to not have loopholes that would break the theory and keep it from working. Those have been resolved.

I AM PLEADING WITH SOMEONE TO BELIEVE IN ME AND CONTACT ME ABOUT GOING FORWARD WITH THIS!
This is real, and I need your help to complete a working prototype of the software and show the world what's been achieved here.

Thanks for your time, I open the floor to some questions (keeping in mind I can't reveal the nature of the idea in full obviously or else I couldn't patent it or keep it safe as my own invention).

What you propose is impossible..end of story.  gauging by your registration date I have to assume you are looking to con some gullible people here out of bitcoins.  If this is not the case, well, then you have your answer now as to why for that long time... no one would listen to you.  Anyways, nice twist on the prince Abubu from Nigeria  with millions he can't get out of the country scam.

Do you want me to invest in your magical money printing company that pays out 50% interest a day?

It's not impossible.  That kind of thinking is exactly why you weren't the one who thought of it.  I did believe it was possible and I thought about it a long long time.  And I achieved it, because I can test my theory and show anyone was interested in starting a company how it works.  I didn't ask for money, and I'm not asking for money.  The first thing I would even want to do is get a legal document signed between interested parties that protects all sides and ensures that if they take on the project, they will get an equal share.  I would of course share the idea with an interested party who signed the necessary documents protecting us all.  And that would happen before any investments took place.  The investments would be made on the belief by that person or persons seeing my idea at work who found it valid once having heard and seen it.  If they didn't believe it would work, and could show me why to my own satisfaction, then they could just walk away.  Where is the harm?

There will always be skeptical non believers to anything, just as there were when the Wright Brothers tried to get their plane flying the first time.  Just as Edison encountered, and Einstein encountered, not that I'm in their league at all, just making a point.

I thank you for your comments and your time.  I'll continue to wait for the right team or right people to see this and take me seriously.  I mean, what hacker or group of hackers wouldn't want to be the first one to solve THIS together?  It's monumental!

Stop being party poopers.

I have the solution for your problem:
Step 1: get a proof of existence of your documents/concept with your name, so nobody can steal it. (I think this was one of the sites, but you should probably do your own diligence: http://www.proofofexistence.com/)
Step 2: Show your unbelievable concept to smart/well-known people who are able to confirm it, get rich, famous and so on.
(In case somebody tries to steal your idea you have the proof existence)

Now you are a Jr. Member, you can post "investment opportunities" in the Lending section. Maybe you will be welcomed by someone who is not a "skeptical non believer" and you will be able to sign the "necessary documents" and whatnot.

I'm just waiting for this guy to post in Lending.. if he dares ;)

Impossible, but you put a lot of work in your post, congratulation !

You evidently have no traits of a professional entrepreneur/hacker according to OP :-)

The theory does not rely on keeping any of the data, that's why I can say 99.8% compression and mean it.  It saves NONE of the data.  I squeeze out every 0 and every 1, and all that's left is a few reference points.  Since binary is only 0's and 1's .... the only thing a binary file is a unique arrangement of them resulting in a program that was created in a higher level language (the systematic formulation of original thought designed to execute a given set of instructions) that are then rendered down into binary for a computer to render at full speed.  If you changed even one bit in the file, it would be corrupted.  So the theory relies on converting the 0's and 1's in a unique way so that only one possible answer is arrived at for the crypto key.  

For example, it's like this ... you want to send your friend a book that he doesn't have.  Instead of sending the actual book, you send a kind of reference guide that tells him what words to look up in all the books he already has.  Now for a man this would take a long time to do, but for a GPU, not so long.  So if the GPU knows how to reference the books in the library already, it can recreate the book from thin air by copying the referenced points given to him in the small crypto key storage file my software program would generate if I got someone to help me do that.  You don't have to copy all those 0's and 1's all the time... that's a sloppy, wasteful system that's gotten us to where we are now, but doesn't have to serve us going forward.  Why send all those 0's and 1's when all you need to send is the thumbprint of the file.  And let the GPU do all the referencing and bring back the original product from it's unique thumbprint?

All I did was think of a foolproof way to do the referencing that results in a very very small code.  From that one code, the program can follow the referencing system backwards and arrive at the answer using brute force.  Once the answer is found, then the entire data set is found with it.  I'm not conning anyone.  I actually love to think and create, but I never had much more skills than that, my imagination has always been my strongest asset.  But an imagination without help is useless, and all I am doing here is asking for help.

Thanks for your time.

I've offered a solution, read my post.
You did want a solution to your problem and not just our money, didn't you?

Okay, if you truly think it's a good idea, Vladimir, I will go and post this in Lending.  Will double posting the same post in another thread get me into trouble, though?  If I will violate any terms of the board by doing so, I don't wish to do that.  If anyone knows that it's okay, I will republish my post in Lending.  Although I don't actually want to get any lending.  What I want is to be a part of a team that makes this theory into the finished reality.  I'm sure lending must come in somewhere along the way, but I'm not asking for that specifically.  I'm more interested in the team of people working with me on my idea and turning it into reality, or discovering that it is indeed flawed.  I truly believe it is possible, that's why I bothered to work on it for 3 years now.  

Let me know if it's okay to double post this thread and I'll post it in Lending. Thanks.

If you don't want to borrow money, don't post in Lending, easy as that.

There's nothing wrong with posting your ideas in Newbies and making a new thread elsewhere when you are a Jr. Member.

Birdy, I tried to thank your for your idea at least three times now, but the board keeps saying "You can't post within 360 of your last post!"  I mean who the heck bans posting less than six minutes apart?  That is truly mind boggling, it ruins immersion in the board chatting experience to have to sit and wait 6 minutes between posts.  I'm sorry but I don't agree with this policy.  Anyway, I thanked you profusely three times and was rejected in posting it 3 times.  This makes my 4th attempt.  And I never copied my post, so I've had to retype the entire message 4 times now. 

So you know I mean it when I say thanks!~

Close the page, wait 6 min, reopen the page and post. Refreshing doesn't do the trick sometimes.
It is a newbie restriction to stop spam.

That would be pretty darned funny ... if that wasn't the same exact joke I get every time I try to explain how Bitcoins are made, and yet here you are, in a Bitcoin forum.  How did you ever get to the point of believing in Bitcoins if you think like that? 

Anyway, if I was in your shoes and I heard someone say this stuff, I would make the same joke as you, but truth be told, it would only be because I was jealous I hadn't thought of it first.

Thanks for the laughs.

haha, damn newb restrictions ^^
you are welcome, at least it's an idea you could give a try.
There are a lot of scammers here, so if anything in your posts even hints at you wanting money from us people will rampage your thread, even more with a concept as unbelievable as this.

Well I kind of thought of this few months ago. And I never thought somebody is thinking deeper about this. I believe this is really possible. And this will not take so much work. Just like the primes or any algorithm or solution set, it is possible to make this happen.

So what kind of team are you looking at? I guess if you needed some more help, contact the developers and not in this forum. I guess some will bad mouth your posts and troll this thread.

I think your really may be on to something here.  For example here in compressed form is an entire book related to the subject of data compression:

ISBN-10: 1846286026

Notice how I have encoded an entire book in a very small number of digits.

http://www.amazon.com/Data-Compression-The-Complete-Reference/dp/1846286026/ref=pd_sim_b_1 (http://www.amazon.com/Data-Compression-The-Complete-Reference/dp/1846286026/ref=pd_sim_b_1)

BTW I have some experience in this area having worked on data compression algorithms in the past.  So, if you want to send me an explaination of the idea I will let you know what I think of it.

I don't 'believe' in Bitcoins. I know they exist, and there is proof that Bitcoins exist.

PS: You'd better not scam anyone. I will be the first to make sure you get arrested :-)

Keep it coming mate. This is what we need. We are now in the age of inspiration. Age of industrialization is coming in its last phase.

Man!  If I scammed anyone on something this cool, that I've actually worked on and created a working prototype theory for, then I'd need to go ahead and arrest myself first!  It'd be a major crime for the entire world.  When you watch Heroes (okay, let's assume you actually watch Heroes ha ha) who do you root for?  Sylar?  Or the good guys?  Because I root for Claire and Hiro and Matt Parkman.  I want the good guys to win.  If I were scamming someone, I would never do it with an idea like this.  This is me at my most earnest.  This is for real, guys.  From the bottom of my heart, I believe I've solved this and only want to see it be born into the world and to take credit for having done the impossible.  I want to feel the pride of believing further than anyone else on something that should have scared away everyone else.  I love this theory, I love thinking about it, it's saved my life on numerous occasions when I almost gave up hope.  I'm basically alive now just to see this through.  Because I have no family of my own, no wife, no girlfriends, just a lot of happy students who think I'm pretty special as a teacher.  That's a great accomplishment in most people's books, but I still want to be famous for doing this one thing.  I won't lie.  I desire fame for creating something. Let's put togethera team and try to DO THIS!

No you haven't.
You have identified and referred to the book, you have not encoded it.
You cannot extract the contents of the book without referring to a full copy of it from somewhere else.


If you have any experience in data compressions algorithms, you will know that they work by targeting particular types and patterns of data.
For any specific algorithm and file size, there is an input file which will result in the compression actually increasing the file size, not reducing it.
And summed up over all the possible files of the same size, compression will result in the same average file size.

Would you care to read about how modern compression algorithms work before you propose one of your own and see if what you are proposing is similar to them. It does sound very similar to me. What you have described is creation of a dictionary and the restructuring data based on this dictionary. The problem with this approach is that it works very well for text data but useless for random binary data including images and movies because resulting dictionary will be of the same size as data itself

Perfect compression is an impossibility.  No algorithm can guarantee that all inputs will be reduced in size.  It is an absolute impossibility.  This hasn't prevented numerous scams over the years very similar to perpetual motion machines or trading algorithms that can't lose.

http://www.faqs.org/faqs/compression-faq/part1/section-8.html

Anyone claiming they can compress all data x% is lying.  Period.  All lossless compression algorithms will result in LARGER file size for some inputs. That is a mathematical certainty.

This is possible.  It can be done.  I would have tried to keep this a secret and work on it on my own, but I'm 42 now, and my brain pan just isn't what it used to be, I can't seem to learn new things very fast, and I can't imagine how to program something like this, because it seems too hard to program.  But the actual theory itself IS easy.

The encoding portion can be done on the fly it's so fast.  So we can get rid of all of those min-harddrives clogging up space in our smart phones (well, we can install apps on them of course, but for saving any other files, they can all be encrypted until its time to view them, when they are moved onto a standard drive for viewing and then deleted once they've been watched, keeping the crypto code in the vault for safekeeping permanently.

We can then offer all cameras, camera phones, video cameras, etc ... all medias that encode data into binary data, the ability to record video in chunks of say 500 Megabytes at a time, each with its own code that is only a FEW Kilobyte's each.  When you want to watch the video, it gets moved and decoded onto the standard hard drive for watching, and then dumped upon completion.

Imagine where we could take this theory if it indeed becomes a reality:

1) No more SD cards for storing large videos.  Recording video on the fly straight into Crypto Key.
2) Facebook and other such companies who do backups could stop having to buy huge server farms that are larger than we can comprehend to store their data.  Back ups would also store very fast, since the encoding portion is nearly instantaneous.  The decoding part is much slower though.  It requires brute force searching through branching timelines of huge data sets to find one "solution" (a needle in a haystack) and that's the only place this theory has its drawbacks.  In the decompression part.  Until we programmed it (and perhaps we could even program it to use Asics instead of GPU's) and saw how fast it could do such kinds of hunts, it's hard to say if it's way too slow to use.  That's my one fear.  We'd have to see.
3) No one could sniff your data enroute and see what it is as it's being transfered.


Thanks for your positive comments, you guys.   I appreciate it.

I'm not lying, I assure you.  If I am in error anywhere, it may be in not understanding some fundamental principle which might SEEM to make this impossible.  But because I do not possess any such understanding, and indeed believe it IS indeed possible, then perhaps I am able to overcome what others deem impossible (and to them IS so because of that belief) while I do not have such a hindrance.  IF you knew the method by which I have achieved this compression, then your claim in quotes above would vanish in a puff of air as your jaw hit the floor.  If I wished to remain a pauper all my life, to prove you wrong, I would indeed tell you the secret right now.  But I've lived a poor life, and now it's time for something to actually go right for me.  I believe I can have a better end to my story than all the starts in it that went nowhere.  

This is not so much "compression" that is almost a misleading concept.  I am placing the entire contents of the data into another container that anyone can access, and that container has a way of recording which 0's and 1's are where in that container.  So that all I need to do is reference to you a code.  You go into the container and decode the references via the software, and the 0's and 1's come back to life as they were originally.  But now that I have the finished crypto code, which is just the master reference key to the sequence hidden inside that container, your computer can follow the instructions programmed in it to solve the block the same way Bitcoins does.  Once the block is solved, you then have the entire original file contained inside that solved timeline.  I know how to do this because it's all I've thought about for 3 years, sometimes for 12 hours a day, sweating myself to death, unable to sleep, because the idea was so cool, I couldn't stop thinking about it.  

So stop trying to negate what is possible by saying it's not possible.  All things are possible if we only knew how.  Tesla's engine is a marvel of the modern that we are still using today, but even Edison didn't believe in it at first, and tried to sell off his faulty vision of creating a motor until at last, the truth of Tesla's engine's power and efficiency was known to dwarf all others.  He knew how to make it better, and no one else could do it like he could.  He had the idea.  Just as I have.

On earth, we call this uploading the file to a server, and giving someone the URL.

But you still have to upload the entire contents of the file onto the server and then also download the entire contents too.  What I'm talking about is rendering the contents into a container contained in nature.  And then the person, using the code, can reproduce the contents right back out of nature.  The actual file never had to go anywhere.  All you needed was the reference itself.  A pointer, if you will, to where to start looking, and when to stop.  It was so hard to think of how to do this, but now that I've thought of it, it's so easy that it actually scares me how easy it is.  Once the world knows, they too will be in awe, just like any wonder of the modern world, or an impossible thing made real before their very eyes.  This feels like magic, but it just logic, pure and simple.

Yeah that is the definition of compression.  Your approach is a textbook scam attempt:
Step 1) Amazing opportunity with no useful facts or details.
Step 2) Build hype
Step 3) Just need some funding and everyone makes a fortune.
Step 4) Disappear.

Perfect compression is a mathematical impossibility.

Okay, it's getting late here where I live, so I'm signing off for the night.  Thanks for all your contributions.  I appreciate your time and energy, even in refuting me.  You are all welcome.  

And please, if any of your know some other Bitcointalk members who might be interested in this topic and my theory, please direct them to this posting and let them talk with me as they may.  This is the best thing I've ever done in my life and I want to do something about it so bad.

Good night everyone.

Like I said I happen to be somewhat of an "expert" on data compression and transmision having worked for 25 years in the disk drive, set top box and video industries (http://en.wikipedia.org/wiki/MPEG-2, http://en.wikipedia.org/wiki/MPEG-4, etc.) so if you would like me to take a look at your idea I will.  I can tell you if the theory looks sound or maybe point our something you may have overlooked.

Are you trying to compress audio, video, text, binary files or all of these?

The OP claims perfect compression of all data in all instances.  When you have perfect compression why limit yourself to one niche?

Well, I never claimed it was perfect 100% compression.  I did say 99.8% .... the reference codes needed to encode the sequence have to be stored into a small file, but that's all you need to send to the other side of the internet and the program on their computer can figure out the rest using the instruction sets programmed into it.  I'm not a scammer, by definition, a scammer is someone who wants others to believe something that isn't true.  Since I am not a scammer and you are telling everyone that I am, by definition that makes you the scammer, trying to belittle my accomplishment, even if it's only in theory.  But it won't ever get to reality if people don't want to help me, which is why your comments can be damaging.  I know their are a lot of cruddy people in this world, and I am not a great man or a man of many achievements, but I care about this idea and about what it can do not only for myself and the team who helps me, but for the world, too.  It's uses go far beyond mere data compression, I assure you.  

Folks, I'm not here asking for money, I'm asking for a team of people, and a contract that preserves all of our rights and our agreed upon monetary valuations as a member of that team (and I get credit for the discovery), and I'll tell you all my idea who is on the team, and if you agree it's doable, we go ahead, and if not, then yes, I disappear at your behest, and no money ever changed hands.  This isn't diabolical, man.  It's a real offer and I really mean to achieve this.  And whoever helps me gets in on the action, that's it.

Even if you claimed 1% compression in all instances it is perfect compression.  Perfect compression doesn't mean 100% reduction but then again even someone with basic knowledge on the subject would know that.


So you will not ask now or ever in the future for a single cent from any "investor" on this forum?

Well, I appreciate your offers and your intention to help, but I'm not going to "send you my idea" and let you do whatever with it...  If you want to help start a team and secure our rights via legal documents, then once those are in my hands, I'll be able to share the idea with you.  But I'm not sending it away into the ether without legal gaurantees.  I've already said as much as I can without getting into the real details that would give away the solution.  Thanks for your interest, though, that does validate my efforts at least in part of trying to create interest in getting this theory turned into a reality.

P.S.:  I tried to leave, but got sucked back in ... hahaha ...  oh, forums!  What they do to us!  Haha.

You want us to believe you have invented a compression method which can compress all files to less than their original size.
That isn't true, therefore by your definition, you are a scammer.

(I disagree with your definition btw, it is more likely that you are simply deluded. A scammer would be someone who wants others to believe something he knows to be false.)

No, I won't ask for any money ever.  I will ask for this:  someone with vision, possibly a future CEO of this company, to gather a group of people who can program very difficult mathematical problems who just like encryption in general.  A lawyer is brought in to help lock in everyone's part and everyone's share of the company based on their skills and usefulness to the project.  Once that's out of the way, I will share the idea and how to accomplish it, proposing my direction for the project from start to finish.  I am not to become the CEO, I don't want power, I want recognition and money, plain and simple.  Once you've all heard the theory and asked your questions, and I've explained it all (this may take up to 2 days before you understand it all and get to the level of comfort with the theory that I have) AND IF YOU AGREE TO GO FORWARD, then the company goes forward.  From that point on, the investors would be there to help us sustain ourselves while programming this thing.  Get a small office or set of offices, the basics.  We program this, we get it working.  We patent it.  OR copyright it, whatever best applies.  That's one for the lawyer.  Once the idea goes to ground and is released, we take our individual cuts of the pie as the software is licensed to companies and the rights to use it are purchased by Facebook, Google, etc ...  we talk to videocamera companies about creating an architecture that allows us to film videos directly to our file format, and now even the smallest watch cameras coming in 2014 can record hundreds of hours of video without running out of space.  We figure out all the ways we can benefit from this discovery.  And that's it.

So where does the money for all this come from (lawyers, offices, corporate formation, patents, etc)?

Please answer the direct question: Will you be asking for money from investors on this forum?  If so then saying you are not asking for money is a lie.

 ???

How is it a waste of time to ask for help when you can't do something yourself?  I can't program software, and even if I did, it wouldn't be efficient like a real programmer who knows what they're doing could do.  Henry Ford was known to be illiterate and was at one time brought before the American people to justify why a CEO of a company couldn't regurgitate commonly known facts anybody knows.  He said "Why do I need to know everything when I have at my fingertips men who know everything?  I can, with the push of a button in my office, have any information I need within moments.  I don't need a dictionary in my head when one in my hands is just as useful."  He was slamming the idiocy of going to school to learn facts that can be found in any book in mere moments, when he was the CEO of a powerful company and could accomplish great things regardless of his educational background. 

By saying I want to waste everyone's time, you clearly don't understand the point of my theory, which is to save everyone enormous time on the internet, cut down internet bottlenecks and peak-time surges that lead to the internet becoming non-useful and annoying in all of its time wasting inefficiency.  My theory is not a time waster, but potentially a huge time saver for the world as a whole! 

I don't want to waste time, I want to create something amazing, with people who like to think and create together.  There's nothing wrong with that.

Ah, too bad BenRayfield hasn't show up here for 4 months now.

WARNING Transactions and Addresses will soon be used as high volume data storage

https://bitcointalk.org/index.php?topic=60386.0

If we could get the "high volume data" kook close to the "crypto compression" kook we could observe a kook-singularity. Or maybe they would just kook-annihilate each other? Anyway, the science of psychoceramics would certainly advance to the next level.

I don't know.
Maybe it's because it's 0200 here and I'm a bit tired, but there's something that just doesn't quite sit with me.

You say you're not a programmer, or have any knowledge of computer sciences, but yet claim to be able to compress and encrypt gigabytes of data instanteously.
I believe you also said that you have been able to do this already and can reproduce the results at will. Then you go on to say that it's all just a theory.

I'm seeing too many contradictions here to believe that you have even the simplest idea of what to do. I don't doubt for one second that you believe this is possible. Unfortunately, I don't believe this feat of engineering is even possible.

Best of luck to you on getting a team together on this.

Ok, I already feel stupid for giving this any more thought, but here it goes...


You say your file will just be a bunch of reference points and then you need a program that then looks up the references to recreate the file.  Whats referenced by the program.  Will be data on my computer, or the internet.. or Nature?  The other thing would be that higher entropy datasets (like images, audio, video) will need many many more references than , say a text file.  So much so that the process of looking up them (references?) could take a very long time , even for a gpu, asic.  So you just need a library of data point with every possible permutation of a certain length of binary data?  Where's that stored?  How fine is each binary data piece is in the library? if its like bitcoin and you are just hashing till you get the right target.. that may take the age of the universe to get the Lord of the Rings movie from a 600kb list of reference points.  What about loss.. i guess this would be loss less compression?  There's to much to go on about.


I'm thinking it'd look  a lot like the early game Pong with that few reference points.


Appealing to the bible for acceptance on faith of your word.. ehh.


..Maybe because after 25 years nothing has come of any of your ideas.  I'm just guessing from your life description.

Desperate, yea... I hope you do too.

Just release your new algorithm under an open source license.
You'll get a name for yourself, and big companies will be knocking at your door for your employment.

OP, please read this page and hopefully you will understand why it's not possible to losslessly compress any data. Some data will actually get bigger: http://en.wikipedia.org/wiki/Pigeonhole_principle

Now I have a _very_ simple example for you. Lets say we want to "compress" 2 bits of data. All the possible datasets are:

00
01
10
11

Now please compress those 4 datasets so that they will all get smaller. Impossible. Unless you use something like:

00 -> 0
01 -> 1
10 -> 10
11 -> 11

But then my next question will be:

Now please compress

0
1

Impossible again, since you've already used 0 and 1 to "compress" 00 and 01. So these 2 "datasets" will have to get bigger by definition, or you will have a collision.

The reason why compression works, is that it encodes sequences and often reoccurring patterns. But as we just saw, there will be datasets (random data) that will actually increase in size if a lossless compression algorithm is used on it. And it just so happens that movies and pictures are very similar to random data. Unless, of course, you use raw pictures/movies, but these files are huge. And believe me when I say you won't beat current algorithms which have been developed over the years by a lot of skillful people.

The thing I think you're missing, is this: You are talking about reference points. This dictionary to which these reference points point, also needs to be stored somewhere. And even if you use referencing, you will still run into the Pigeon Hole Principle. 2 bits of data have 4 possible combinations, so in order to be able to reference to those 4 possible combinations you need 4 different reference points.....Which require 2 bits of data.

Hope this helps. :)

You claim to have broken Kolmogorov complexity (http://en.wikipedia.org/wiki/Kolmogorov_complexity),which is so huge,that it can be the discovery of this century.

I had to tried and imagined once to use complex mathematical functions to generate entire bit sequence of a huge file (some complex function % 2),but theoretical its near to impossible for every file,as every file have different random sequence of bits.

As you say you had a working prototype,if you are correct than prizes are waiting for you Hutter Prize (http://en.wikipedia.org/wiki/Hutter_Prize).You should send your entry there and when you will win with a a seemingly impossible improvement,instantly the whole world will recognize you :)

Mushoz, I am impressed with your attempt to help me see the error of my ways.  It is a valid argument, and its one I encountered when I was first studying compression in the first place.  But I didn't want to do someone else's compression Idea, so I stopped studying compression theories after the first 30 days, because I realized all I could do was try (and only TRY) to improve a few percentage points over what had been done before, and my math skills were not up to par for that kind of challenge. What I was seeking was a kind of Einstein-like E=MC(squared) formula that was small, efficient, and could do what needed to be done.  Imagine trying to copy a jpg image.  Every bit is unique in its composition and arrangement.  How could you retain that fingerprint without actually compressing anything?  I am not replacing any bits, I am not trying to store that 1 and 0 you claim I am trying to store.  I figured out a way to store every unique placement of every bit in this example jpeg image within a natural container open to all peoples of the world, that would create a unique truly 100% imprint of the data in that file within that structure, and then give me a pointer that I can then teach a software program to decipher (using logical binary rules) how to SOLVE THE BLOCK like any crypto currency does.  Once the block is solved, the solution IS the dataset used to solve the block!  It's absolutely awesome, I am literally in awe of the idea itself that I've arrived at.  I want to share this with the world.  I am going about this is a totally new and unique way NO ONE has ever thought about before. That's why it's not compression like we've ever known it before.  The only reason I say it's only 99.8% complete is that when my (future) software program saves out the references it needs to be able to logically solve the block when the time comes, there are a few bytes of that per every 10 Gigabytes or so that have to be recorded to ensure recovery of the block later.  All of the entire contents of our example's jpeg image goes into the solution, but what comes back out are the crypto codes and reference points adding up to about 0.2% of the overall file size.  If computing power was fast enough, I could gaurantee recovery of any size file with merely one crypto code, but at the present time, I cannot gaurantee fast enough speeds, unless we use a quantum mbit computer.  Then yes, with that computer, I could say my file compression is 100% whether you believe it or agree with that conclusion or not.  Yes.  That's what is possible here.  My idea, given a few years of maturation on the world market, could indeed compress all data to nothing if the computing power was strong enough to decode it without taking a lifetime to achieve it.  Once I publish this, you will read it for yourself and see for yourself and marvel at it as I do every day.  So I am very thankful for those of you are sending me positive messages and PM'ing me about offers to help. This is already beginning to become helpful, as I had hoped.  I think the world needs this idea badly, and I don't think anyone or anything can stop it from eventually coming out.  I'm just trying to be the one who gets remembered for inventing it.  I have never amounted to much and except for a few students who speak fondly of me and have tried to get me some teaching awards, I haven't done anything to make my family proud of me.  One day, this will be my legacy.  So I need to stay attached to it.  But if I get cancer and am going to die or anything, then yes, I'll go ahead and come on here and give it away in the interest of science and the future of mankind.  I won't go to my grave clutching the secret, I assure you.

Einstein did know his math.

I have see this before.

Bored one day, I invented a "new math" that did not use addition, subtraction, multiplcation, or division signs. I called the new sign a "fordor". It works by simply concatenating the two numbers around the operator. For example:
1@1 = 11
22@43 = 2243
1@98 = 198

An entrepreneur type who saw me working on this said that it may be useful for data compression in computers. Even as a child, I knew that this was not the case because any compression algorithm based on my "fodor" concept would be "lossy". Look at the '198' example above: it could be the result of 19@8 or even 198@.

However the entrepreneur type ran with the idea despite my objections. Perhaps he was thinking along the same lines before seeing my work on "fordors". I later learned he applied for a patent on the idea. He wrote software and produced retail boxes. I was told one example he liked to use is that a whale comes from a sperm and and egg.

The software never sold well because it never achieved it's 90% compression claims. In a review, PC Magazine said none of their test files resembled the originals upon decompression. If you don't believe me, I may be able to find some of these references, but it would take some digging.

The point is that ideas like this are common. Ideas are a dime as dozen. Implementing ideas is the difficult part: and you are not going to do it by isolating yourself for fear that somebody is going to steal your big pay-day.

Afraid someone will steal your idea? (https://opensource.com/life/13/8/stealing-ideas)

Re the kook comments: kooks have a niche. They can force you to think about problems in new ways to explain why an idea won't work. It has in fact been explained in several ways, the OP does not appear to understand, why it won't work.
 

Update:  

I received a PM asking that I demonstrate my ability to beat the Kolmogorov Complexity by achieving 1st place in the Hutter Prize.  Since I have said that my compression technique can achieve 99.8% of the file... winning the Hutter prize requires creating an executable file (standalone) which can recreate the exact file placed at this URL:  http://prize.hutter1.net/   in under 1 Gigabyte of Memory and 10 GB of hard drive space.  If a programmer could program my the executable decoding program from my theory at a size of 1 megabyte, then I propose that my complete file size with the program included would be less than 1.5 Megabytes, which would qualify the person who helped me program this to at least the Grand Prize of 50,000 euros.  The prize awards 500 euros for each 1% of improvement.  We would be giving them FAR more than just 100% if our file size was only 1.5 Mbs.  The Hutter file to be encoded is 100 megabytes, and one man has gotten it down to 16 megabytes.  Imagine what they would do when they saw 1.5?  hahaha!

Maybe that is enough incentive to get someone amongst you interested now.  Maybe just a small team, 2-3 people.  We can do this.

A million times, this.  True for all ideas, no matter what field you're talking about, technical or creative.

Just a few encouragement, I had a business partner before that is an artist. He loves to paint, and make arts. And he has a hard time doing math. But instead of being discouraged, he used his strength(creativity) and made a software that would run a stores point-of-sales system, so what happened was he made an excel, yes excel as in spreadsheet using the macro and from there he made the spreadsheet run the whole pos and even networked it!

So heads up!

You'd better keep your word then. I have all the information I need about you, Christopher.

If the file is 100MB and you reduce/compress it down to 1.5MB, that is 98.5% reduction.

I'm not sure in what deluded fantasy math you've invented in your head that 98.5% = "FAR more than just 100%", but you're a loon.

You don't need a team of programmers, you need a team of psychoanalysts to get inside your head and fix what's broken in there.

If you want a dime from anyone you publish your amazing idea, in detail step by step explaining what you do. You put your name on it and with the fame and recognition that comes from being the inventor you rise to glory.

If you are half as smart as you are telling us you are then you shouldn't have any problems understanding the value that would give you and how it would rocket you from nobody to somebody and land you your dream gig.

You wouldn't be posting on bitcointalk looking for a handout and you wouldn't feed us some BS emotional story.

I'm sorry you feel you need to resort to blathering to achieve your points, but no one needs degradation as a form of critiquing.  I don't need it, and you don't need it either, so let's keep this civil.  I appreciate your interest in my post, however.  

Numbers like what my compression ratio could be once the software is completed is pointless now.  All I can say is, for large data, like 100 Gigabytes, that kind of guess on my part is really just a guess (99.8%) at this point, and is not worth the getting all wound up over.  For a file the size of the Hutter contest, 100 megs, I am fairly certain the brute force method can still apply, so internal file splitting to achieve multiple crypto keys would not be needed for a file that small.  Thus, I am certain I could achieve 100% compression of that file, if you want to call it compression.  It's not really compression as society knows it today.  It's a way of hiding the data and then knowing how to bring it back from a known natural container.  Bringing it back requires brute force methods, as far as I know.  For 100 megs, if you want to split fine hairs, Aahzman, the final crypto key would like (not exactly like but something) like this:   (xxx.yyyyyyyyyy.zzz)  Where each other those three points are required to tell the decoding engine how to work out the details and solve the block.  Since only one crypto key would be needed, whatever amount of data that is above (3 x's 10 y's and 3 z's = 16 characters x 7bits = 112 bytes) ... so maybe for 100 megs, only 112 bytes would be needed.  But for larger sizes, I couldn't yet say until it was explored by my team.


Thanks again.

If I follow your advice here, I would write out my idea, post it somewhere and wait for glory to come.  Meanwhile, someone else with money would take my idea, actually formulate a software program with it, patent that, and since he came to the patent office first, he would get the credit, not me.  I don't see how you are thinking straight on this.

P.S., my life's story is not BS (not to me at least) .... and I didn't know you found it emotional, I appreciate the hidden compliment regardless of whether you meant it or not. 

PPS:  How is it to be called a "handout" ... a handout is where you take something for nothing and don't give back in any way.  Words are powerful, please choose yours more wisely.  It's nothing like a handout to ask for people to help join a team and work out a new creation into the world that could be of great importance.  And last but not least, I'm not claiming to be smart, in fact, I don't think I am very smart actually.  I do submit that I am creative and not willing to say "that's impossible" or let negative comments from the audience discourage me from my theory and from asking for the help I need, there is no shame in that ....  if you aren't Steve Jobs or Bill Gates.  Both are/were great businessmen and knew how do amazing things in business.  That's never been me, sorry to dissappoint you.  I wish I had the savvy to do it all on my own, but I don't.  And I won't let that stop me from asking for help. 

Thanks anyway.

Text :

Binary : (space between two bytes)

Can you just show your compressed output for this 96 byte binary data ? A sort of demo .

Creativity is different from intelligence.Countless intelligent people saw the things falling down but Newton observed it.The thing is that, small things that go unnoticed may solve the seemingly impossible.

If you compress a 100 megabyte file down to 112 bytes then that would be really cool.

I have only one small question for you:

If I change one byte in the 100 megabyte file would the xxx.yyyyyyyyyy.zzz crypto key change?

I am going to assume it must because you need to be able to recover the two different (different by only one byte) 100 megabyte files, right?

Hi, Burt, after reading your resume you sent me, I almost fell out of my chair.  You not only have every qualification needed to do my project, you also enjoy knowledge of working with videocamera microprocessors.  And since one part of the theory is that once we get this working, we will be able to record video directly into the crypto code format at under certain size limits (say 1 GB) and anything over will be a tad more complex, but only a tad.  I am very interested now in sharing my theory with you.  You must have worked with NDA documents before.   Let's discuss contact information in a PM post and get started on sharing this with you.  I will of course record my documents in the crypto block chain using Proof of Existence first.  Then we can chat via Skype.  You will have to help me formulate the correct terminology to use in describing this with you.  I tried before with one of my best friends (who was a computer programmer for Gas Powered Games) but my "creative terminology" insulted his multi-talented logical brain and he couldn't "get me" ... I don't want that to happen again.  

PS:  If you change even 1 BIT anywhere in the 100 megabyte file from your question, then yes, the crypto key changes to something else.  It's a unique DNA-like signature of the entire contents of the file, so every last bit in necessary to decode it, making it truly lossless.  I can almost feel from your question that you are already close to understanding where I am going with this (and that both exhilerates me and frightens me hahaha).  


Spndr7, I would need a sort of mini version of my encoding program to be able to answer your question regarding the 96-bytes of binary data.  I can't do it in my head, and I can't do it on paper with that much data.  I can do about 10 bytes, but even that would take like 12 hours of work by hand. This kind of work is exactly why computers were invented in the first place.  The good news is that I know how to program in basic, and with the support you are all giving me, perhaps I should try to program just the encoder part so that I can answer your question to a certain level, say to at least 1024 bytes of data.  So I can share with the people who are interested and sign the NDA's and wish to help me.

Would you share how long it would take you to do it on paper with 9 bytes, 8 bytes, 11 bytes. I want to see how complexity would grow with increased data sets.

Edit: and how big the result would be for those cases (8,9,10,11 bytes)

Time scale by Hand to get a Crypto Code:

Approximately 30 minutes for each byte due to double checking.  If even one number is done wrong along the way, the whole process is ruined.  Double and triple checking is a must.

Around 6 or 7 bytes into the process, you begin to get paranoid you've made an error, because by hand, it's easy to do.  Sometimes you have to start all the way over, and go all the way through to 6 and 7 again.  In that case, 1 hour for each byte.  My friend and I spent hours just checking and rechecking every digit in our trial runs.  It was fun, but exhausting.  And that was back when I hadn't optimized by theory down to a few rules.  It had so many rules in the beginning, it was impossible to understand.  But I knew what I was trying to do, I just hadn't asked the right questions yet.

Anyway, the double checking along the way is why I said it would take so long to do 10 bytes by hand.  If done by a computer, the encoding would take 1/1000ths of millisecond (I'm joking guys, don't write flames over this number, it's just some comedy relief at this point.) 

Thanks for your interest, guys.

So you are guessing. If you read between the lines in Rationale for a Large Text Compression Benchmark (http://mattmahoney.net/dc/rationale.html), you will see they have an estimate of how much compression they expect is possible. The estimated entropy of the text, together with the pigeon hole principle (mentioned in this thread), puts an upper bound on the maximum compression that is achievable. That page estimates about an 8:1 compression ratio; assuming 8 bit characters (Wikipedia uses UTF-8). They even include a proof that a program can not be any shorter the length of the shortest possible program is not computable.:

I have seen cryptographic hash functions described as "compression" in the documentation for Freenet. The SHA-256 hash of the test file in that contest is:
2b49720ec4d78c3c9fabaee6e4179a5e997302b3a70029f30f2d582218c024a8  enwik8

However, that hash is not sufficient to reconstruct the original file: even with brute-force. Leaving aside the calculations that brute-forcing sha-256 would take more energy than is in the known universe, it is still not enough. The difficulty is that SHA-256 gives you a string of a fixed length. As with all hash functions, there exists an infinite number of inputs that will give you the same hash. So, even if you use "brute force" to find an input that gives you the same hash: chances are high that it won't actually match the original. You can try to improve the odds by making sure all of your guesses are the correct size. The pigeon hole principle says that even if you generate 100MB of noise: you only have to change 32 bytes in the file to generate a collision.

Here is the size for 1 byte ....   (x.yyy.z)
Here is the size for 5 bytes:     (x.yyyy.z)


Here is an intriguing thing about this theory.  If we get it working, for files that use no splitting internally (maybe 500 MB or less) there would only be one crypto key.  If you could remember it in your head, you wouldn't even need any file!  Our program will have a crypto key direct entry mode.  You type in the crypto code from memory and your file is restored.  Why?  For total legal security of your documents!  

Under American law, if you go through the airport, anything you carry with you like your computer can be legally opened and searched.  If you put your iphone inside of a metal box with a key, that key can be demanded by the government because its physical.  But the law at this time won't allow the government to force you to give the contents of your mind.  So if you use a lock that requires a password, they can't make you give them the password by law.  Imagine zipping your top secret documents and then compressing them using our software.  Let's say your files are 450 megs, and our software's cut-off point for a single crypto key is 500 MBs.  So you get your crypto key, easily memorize it, and now you are carrying the entire 450MB file IN YOUR MIND.  Its like Johnny Mnemonic, only better, because all that data isn't actually in your mind!  When you go through the airport, you don't even need your computer, the top secret files are in your mind, safe from badguys, safe from the law should they try to use it against you.

So then you get to where you are going, download our software onto any computer you want, and use the "crypto key direct entry" mode and type in your crypto key following the format (x.yyyyy.z) and your document is recreated out of the ether for you!

It sounds like science fiction, but it just makes me feel exhilerated beyond words!  I can't wait to see this work.  It will be a true marvel.

In that case, the average amount of space taken up with your crypto key will be the same as the average size of the files you are compressing.
That is just a simple fact.
There are 2^32 possible files of 32 binary bits length, they are all unique, and therefore they cannot all be indexed by less than 2^32 bits of information.

Let me guess, your method is something along the lines of computing some sort of hash of the file, and then to decompress, you hash every possible random block of the same length until you find one which matches the hash, and hey presto, you've reconstructed your original file?
That doesn't work, as there are many many different files which would all give the same hash value.

I believe you, and I know you know what you are talking about.  But the flaw in all of this is calling what I am doing "compression" ... it is not the changing of the information into a smaller form using a system that throws out predictable information and then lets it get added back at decompression.  What I am doing is copying the contents of the file into a virtual container, thus the entire file is actually still there.

I don't wish to have to do this because others will again be calling me a kook, but I fear I must.  Imagine a virtual space (like say another dimension) as seen for example in the Harry Potter movies where the girl's hat always held whatever she needed and she could pull it out at will.  It still existed in its original form but was hiding in another dimension.  That would be a more likely description of what I am doing here.  Shoving the file into virtual reality that exists in nature, if you know how to look at nature that way.  The code you get is a 100% logical operand that can be traced back to the original file source.  Once that is solved, the entire "blockchain" as it were IS the file itself, hidden in this virtual container.

That's why talking about compression schemes cannot begin to rationalize this.  You have to think outside of the box totally on this one.  It's an entirely new thing.  That's why I doubt those behind Hutter's prize would actually pay out for this ... because what they are trying to get the world to do is create smarter compression engines that beat the current ones in a mathematically smart way, so that intelligence can contribute to the world.  I would still like to try, but my feeling is when they see how I did this, they will disqualify me to win that contest, even though the achievement itself would be quite remarkable.  Plus, I am not sure I can get this to work with less than 2 to 3 GB or RAM, which would also disqualify this.  And, there is no way to know if what I am proposing will not require an infinite length of time to decompress the file.  That also has to be addressed first before we would know how good this is.

Again, thanks guys.

A judge certainly can order you to give it up with just cause, and can jail you for contempt of court if you fail to comply. So maybe not 'force', but they can find enough other reasons to make your life miserable that you might be compelled to give it up.

My goal is to compress large files, not small 10k files.  The smallest file I would even want to compress with this theory would be 1-2 megabyte(s).  Small enough to compress some jpeg images into a zip file then take that and compress it through my system into a 10k file to send to your friends.  I don't want to use this compress small text files, that would be senseless.  I'm talking about using this to encode 50 GB videogames for sending over the Playstation/Xbox networks at virtually no download cost or speed.  I'm talking about sending Blueray movies from subscribers to movie websites which offer movie rental, where the true Blueray (full-sized at 50GB) would encrypt to under 500Kbs for quick download by customers who still had sub-Megabit internet speeds.  I'm talking about people hoarding their video collections in 500 MB containers and being able to go to a friend's house, type in the direct crypto key, retrieve their stashes out of thin air wherever they go to show their friends.  The world will be fundamentally changed should we be able to pull this off.  If not, I'll be the biggest laughingstock who ever lived, I suppose.  It's a risk I have to take, though.  Because I like cool things and this would be a lot of fun to see borne.

Well, first you have to prove mathematically that your method works for small data sets. The main premise for you to prove is that multiple files (data sets) wouldn't results into the same sequence of xx.yy.zz. It would be easy to prove:

Just calculate it for all the numbers from 1 to let say 2^32 and see if there are any collisions. And if the resulting string of the 4 bytes (2^32) sequence is less than 4 bytes, you will have collisions due to the pigeon hole principle mentioned above.

That is science fiction.
I download your program, zip and encrypt my 487MB files, and memorize the crypto key of xxx.yyyyyyyyy.zz.
That's now all on my laptop. It's safe. It's secure. It's not going anywhere.
When I reach my destination, I cannot simply download your program to any computer and type in my key. The information is stored on my laptop 1500 miles away. Not on your server. And if my laptop is powered down with the battery removed, then the information is completely cold/offline, and impossible to retrieve from a distance.

I would suggest seeing a psychiatrist. That is a professional opinion.

http://en.wikipedia.org/wiki/Fractal_compression

Basic information theory says it is not possible to produce an algorithm that reduces the size of every possible input sequence. No matter what magical algorithm you are using, I will be able to construct input that, when compressed, take up as much or more space than before.

If you use a magical "dna sequence" (hash value call it what you will) with a length of 128 bytes, that gives you 2^1024 possible ways to represent a file. The problem is that I can construct 2^1024+1 different files, and it has to follow then that two of these files will produce the same hash value. You will have no way of knowing which of these two files was used as the original input for your algorithm.

There is no "loophole" or "alternative dimensions" that will allow you to represent 2^1024+1 unique values with 2^1024 bits.

-Michael

No, I'm afraid you'll be just another internet kook noone had ever heard of.

Release it on an open source license then.

FIFY

Generally if you understand how computers work... and what exactly "32-bits" represents - aka 4billion possible combos.  You start to get a very VERY crystal clear picture of how hard compression is.

To be honest, most compression now adays is for very very specific types of data (mp4/mp3/jpg - and "lossy").  General multipurpose true (as in what you compress you get _exactly_ back when you de-compress) - compression - IE zip/rar/etc aren't very powerful on the grand scale of things.  They just remove "wasted" space.

@Basic miner

Consider putting your idea also on Encode.ru (http://encode.ru/) the biggest data compression forum.The winners of hutter prize are its members and regularly post there.

Everybody will laugh at him :-)

Michael, thanks for your comments, but you don't know what I am doing and I can't tell you everything of course.  But the method I am using to copy the data allows for infinite possibilities, where the file's size actually helps to make it more unique than less unique.  The larger the file, the more impossible another file could have its "hash" as you call it, although I am not using hashes because that implies a mathematical compression algorythm.  And it's not even an algorythm.  It's just three core rules.  It's so easy to do the encoding, it could be done on the fly by even the world's slowest CPU, even for large data sets.  It's the decoding that will take fast GPU or ASIC rendering to solve the block the way Bitcoin does.

PS:  There are containers in nature (into which data can be stuffed if you know how, and I do) and it can be downloaded from that container anywhere in the world by anyone if they knew the crypto code.  What this says is that everything we've ever done is already contained in nature if you know how the bits were arranged, all possible combinations already exist in nature, not only for everything we've already done, but for everything we will ever do in the future, too.  But it takes understanding how my theory works to be able to realize how that.

You should waste time writing lengthy posts, and code test implementation of your idea instead to see how it does work  ;)

You certainly are swimming against the main stream here.  I know you keep saying that you are not describing a compression algorithm but you need to know that all of the current modern video compression standards fully specify the decoder and leave the encoder unspecified.

You are saying that you are going to specify the encoder but leave the decoder unspecified?

Specifying the encoder and not the decoder is kind of like saying the answer to life, the universe and everything is 42 - we just don't know the question yet.

Is there any way you can specify the decoding process?

Look, I'm sure you put a lot of thought into this but you are out to do something that just isn't possible. It doesn't matter if its a hash value or if the numbers come from a magic 8 ball or though visions by psychics. The simple fact is that if you have 4 bytes compressed size, then the total number of unique inputs that can be represented by those 32 bits are 2^32 = some 4 billion or so. But I can create a lot more unique files than 4 billion. You have to see that. Decoding the 4 bytes compressed data has multiple solutions (actually an infinite number of solutions) and there is no way to know which one of these solutions was the original input.

You're trying to build a perpetuum mobile and refuse to accept the mathematical and physical facts that tell you that this just isn't possible. Sorry, don't mean to be harsh, but you are wasting time and energy.

-Michael

How is it a waste of time and energy to scam money off of "investors"?  Your only mistaken is in thinking the OP is genuine.

BurtW:
The decoding method is Brute Force, sorting descending columns of binary branches looking for the blockchain that solves a timeline.  That's why I need some help programming this, if only to find out how long such a routine will take, because it could be that although my theory will work in theory, in reality, the time needed to decode the block may outweight ever actually using it.   Then the theory is busted anyway, due to that possibility.

 
Michael:
Has Pi ever been solved?  Because I was under the notion that it was an irrational number of non-repeating digits going on endlessly.  My theory requires the use of Pi to at least 1 or 2 GB which are held in memory.  The bits of 0s and 1s create a branching storyline through Pi that, based on my ruleset, can only arrive at one location in Pi for every unique file.  If that is the case, how is it possible that you can say my Crypto Key isn't unlimited, or INFINITE even?  The index location itself that you arrive at in Pi is the key.  You only arrive at the location you arrive at by following a few rules.  There is NO POSSIBLE WAY to arrive at any location with two different files because the bits inside those files will cause the program to take a different turn when encoding it, arriving at a different place in the "timeline" of Pi.  If you took the same jpeg image, loaded it and changed one pixel from white to black, saved it out, so that only 1 pixel was different, then that change would ripple through the timeline, meaning you would stop on a totally different place in Pi.  We can't possible go too far into Pi because we have to be able to sort backwards from the Index we stopped at and find the decimal point in Pi 3.14 etc...  the decimal point is the beginning point.  Our timeline must fit exactly to the last digit, otherwise, the program keeps hunting through the possible combinations.  That's why I say it could end up being that my theory can't work based on the time needed to search billions of combinations, although the actual searching isn't encrypted, it just straight numbers, so it should be pretty fast, I would wager.

Now you've gone and gotten me to give away a large portion of the concept, I hope this doesn't come back to bite me.

And so is tesla considered to be mad and crazy. But look, his systems do work.

First, please stop calling it a blockchain - it is not a blockchain or you do not understand what a blockchain is.

Secondy, can you apply your method to a sequence which looks like that xxx.yyyy.zzzz to reduce it even further? As you claim, that you can compress any data, it mean you can further reduce its size, right? So a string in a form of xxx.yyy.zzzz could be reduced to xx.yy.zz, and why stop there, let's run it again and produce an even smaller "crypto-hash" by running it again to look like that: x.y.z. What is stopping us? Do you see what I just did? I just improved you method from 99.8 to 99.99998 compression ratio. Don't you see a contradiction there?

root 2      1.41421356 23730950 48801688 72420969 80785696 71875376 94807317 66797379 ...
binary         01001110 01110110 00001000 10000101 00101010 11011110 10001111 00111111 ...


Here 8 byte binary data can be represented by modding every digit of root 2 (0 for even and 1 for odd) by 2 till 64 places(size of data).

I don't know whether its possible or not to finding corresponding decimal digit sequences of irrational numbers for arbitrary data set.
Mapping meaningful data to nature's randomness would be too difficult.
It is seems impossible.What do you say 'b ASIC Miner' ?

It is not impossible, and it's not even difficult, because I have done it.  At least in theory (which I have proven on paper at least).  I still have to wade through actually implementing it with the right team of people to see if it actually works in reality not just just theory.

Again, there is a trade off for doing this, it means we have to do a huge amount of hunting through huge strings of data over and over again, and I'm not sure it won't take 25 years to solve one file if it's over 1 GB in size.  But I've heard those Quantum 500 mbit computers can do calculations that boggle the mind.  Of course, then only the government could use my theory in that case.  I'm hoping that this theory will work for the average user, or else it isn't something I want to even create.  Just more power to the government, I don't want that.

The smallest text file size is 4 Kilobytes sitting empty on your desktop.  If you fill it with a block of text and save it, it still read 4 kilobytes.  In that text file would be something that looked like this (I'm not a programmer so this is only an illustration, please no criticism unless its to actually teach me):

[OPENFILE]
[filesize=2400000000_&_Name="video_for_kslavik.avi"]
[MChunks=4_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[2,w, xxxxxx, y, zzzz]
[3,w, xxxxxx, y, zzzz]
[4,w, xxxxxx, y, zzzz]
[CLOSEFILE]

If you were to run that through MY theory (which is made for larger files above 1 megabyte) you would still end up with this for the next smaller part:

[OPENFILE]
[Layer=2]
[MChunks=1_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[CLOSEFILE]

SO WHAT WOULD BE THE POINT?  I suppose you could do a RAR compression of the text file, but still, what's the point?  It's already small enough.

It doesn't matter.
If your index key is only N bits long, you cannot index more than 2^N files. No matter how much more you post, this will continue to be true.

Pi is an irrational number, which means there is no fixed sequence of recurring digits which continues indefinitely.
It does not mean that every sequence of Y digits is guaranteed to be unique.
In fact, even a few moments of thought will tell you that there must be repeating sequences, otherwise Pi would be representable by a finite string of digits, which it isn't.

So there is a possible way to end up with the same index key from two different files, and that is how you end up with only 2^N possible answers.

No.
The file will always take up at least one filesystem block, and it may be that for your filesystem each block is 4Kb.
That does not mean that the file contains 4Kb of information.

I appreciate you taking the time to explain matters, but I haven't your background, and I'm not sure what you mean by 2^N, so I can't actually formulate any response to your critique that will make sense to you.

Every unique file is, itself, made up of repeating strings of data, but what makes the data unique from other files is the arrangment of the data, how the 0s and 1s are arranged.  Thats true for almost everything. Some popular music differs from other songs only by a few notes and the main singer, but we view almost all of it as unique, the small differences do a lot, even if its almost the same.

What I've created is a way for every bit (0 or 1) to affect the path taken to reach the index in Pi.  Therefore, its impossible for any index in Pi to hold more than any one outcome.

Take 0000001  and    0001000   and  100000 for example.   The index for each is, respectively:

BYTE EXAMPLE:              0000001:       0001000:        100000:
    Pi Index:                      (57)               (85)             (103)

And that's just with one byte of data.  The larger the file size, the more unique it becomes.   You could have billions of 1 megabyte files, and if even one bit in their internal structure was different, the outcome would be different.  This is truly the butterfly effect at work.

Lets go in stages:
a) If you have billions of unique files, they must map to billions of unique Pi indexes, otherwise two files would map to the same index, and you would not be able to to determine which of the two files was the original. Do you agree?
b) If you had hundreds of billions of unique files, they must map to hundreds of millions of unique Pi indexes, otherwise two files... Do you agree?
c) Therefore the number of possible indexes increases as the number of possible input files increases. Do you agree?
d) Therefore the number of possible indexes increases as the size of the input files increase. Do you agree?
e) Therefore there is no fixed length of index string which can possibly represent all of the possible indexes for all possible files. Do you agree?
If you have got all the way through to e) and still agree, then you should see that your scheme as presented, with a fixed index length which represents all possible files, is impossible.

To take a small example, lets look at all files of 32 bits in length.
There are 2^32 of them. (2*2*2*2... 32 with 32 numbers 2s in).
That is 4294967296 in decimal.
So there are 4294967296 possible input files.
They each need to have a unique Pi index.
Therefore there must be 4294967296 Pi indexes for them.
That is 2^32 Pi indexes.
Therefore you need 32 bits to represent the possible Pi indexes.
Therefore, as explained before, the average file size after compression will be the same (or greater, with a poor compression scheme) that the average input file size.

Doh!  I finally understood your meaning, sorry.  I do see your point at last.  But that only helps me in the long run, because now I can do something to implement an offset function, meaning we can reset where to start from in Pi if and when we do run into such an error during the research phase of programming this.

I am working with Pi, but I could just as easily work with other irrational numbers, too.  This would serve as a way to encrypt each file by offering hundreds of possible irrational number sets so someone couldn't easily find your file if they somehow got a hold of your Crypto Key.

Isn't it possible we could work with a large enough index key that the N bits you are referring to are larger than the number of files in existence at present or any time in the near future?  How many total files are there in the world at this point?  Does anyone know?

So if the sequence of bits you are looking for is very long:

   01000010001001000...101001000111

Let's say 1 million bits then are you trying to locate this 1 million bit sequence in pi?

You still don't get it, do you:  if you take all 64 bit files in the universe (8 bytes), there are 2^64 possible files which could be build with those 8 bytes (only 8 bytes). How are you going to represent all those 8 Quintillion files with an index which is less than 8 bytes?

Here is a very simple example.

We know pi out to who know how many digits - for the sake of this example let's call it "enough".

Your phone number is 123-456-7890

You find your phone number and it happens to be at location 987,654 in the number pi.

You can say "My phone number is 123-456-7890".  This takes 10 digits.

OR

You can say my phone number is at the 987,654 digit of pi.  Cool!

BUT

What if you find your phone number at location 987,654,321,987,654,321 in the number pi.

You can say "My phone number is 123-456-7890".  This takes 10 digits.

OR

You can say my phone number is at the 987,654,321,987,654,321 digit of pi - but this actually takes more digits.

Have you solved this problem?

You could pick some arbitrarily large value of N, sure.
But the second shoe, which hasn't dropped yet, is that your index key will (on average) be the same size as the files you are trying to compress.
So it takes up just as much space (on average) to store the index key as it would to store the file itself.
To compress 1MB files, you need a 1MB index key.
To compress 100GB files, you need a 100GB index key.

BurtW, no.... I am not trying to locate sequences of digits in Pi.  I've heard of that in my own research and felt that was not the way to go.  I was told hunting for strings in Pi couldn't possibly work, and it also would have required math that I am not capable of in the first place.

Pi is an Index that we move through as we are encoding.  Movement is based on whether the digits are 0s or 1s.  Movement occurs in 8 bit increments (one character at a time), therefore all data must be read in its native format and if not in binary format (such as hexidecimal code isn't) it must be converted to Ascii Binary.  Starting from the decimal, we read forward 8 bits in per character.  I have tested this and the general rule is 100 indexes per byte.

murraypaul, I understand that you are very smart and that you know how logical math works, but are you going to sit here and tell me that every one of those possible combinations of 64-bit files are arranged into meaningful orderly arrangements?  No.  Most of the arrangements are going to be nothing but white noise.  If I go and create an orderly 64-bit image, I'm not sure that image wasn't already ordered in Pi to begin with.  There might be 4.29 billion unique 32 bit files possible, but are there actually 4.29 billion such files in existence now?  Could there even be?  I doubt it.  How much of 4.29 billion unique combinations did the Nintendo era actually use in creating its games?  Probably 1%.  And then we moved on to 64 bit games.  And then we moved on again.  We never sat too long in any one size before expanding the sizes we needed and creating vastly more incredible works.

Besides, I know my theory won't work with one long file because I believe it would take 100 years for the CPU/GPU whatever to actually find the sequence going from the Index End point in Pi backwards to the decimal point, verifying that it's the one unique path to the start of Pi after the decimal point.  So I would be working with 500 meg chunks, called Mega Chunks.  For every 2 GB of data, there would be 4 Mega Chunks for example.  For Crypto Codes, as my example above for the video file represents.

Vladamir, what is your trip?  I swear, sometimes you programmers can be so negative about everything and just shoot everything down without caring that there is a person on the end of your scathing words that is trying to do something cool.  I don't need such nonsense here.  Help me or stay out of the aisleway.  I'm working here.

BurtW, come on man.  This is funny but only if you are not really trying to mock me.  I get the humor, but still. 

I've said before, and will say again, this is not for small compression of under 1 megabytes of data.   Since I am also assigning one Crypto Key per set amount of data (for example, 500 Megabytes unless that doesn't work so well) each Crypto Key for files larger than the set size would have multiple crypto keys, meaning combinations of crypto keys, further adding to the complexity of the overall N bits of Index, since then we would be combining Crypto Keys into new patterns as well as relying on a singular Pi index.  I am beginning to understand what you are telling me about possibilities, but you also must know that not every possibility contains a valid thought-organized answer, some would always be random noise.  Perhaps Nature has a set limit on the amount of creativity possible, which is why evolution causes us to change in the first place, so to avoid a bufferoverflow error of some sort.  (Okay, that's my own attempt at a joke).

Which is why compression programs like zip and rar generally do produce smaller files, because they target particular patterns of data which tend to be seen.
General purpose compression routines can only get you so far though, which is why there are specialist audio and video compression routines, which target the common types of data seen in those arenas.
But you started off with a general claim about being able to compress all files. I think you now understand that you can't.
So now you realise you haven't got something magical, the next question is: What makes you think your method is better than those which already exist, and have been comprehensively studied for years or even decades?

But where is that space at?  It's in memory.  Not hard drive space.  The key gets expanded to the 1GB or 2GB size limit, sure.  Then you encode your file forward in memory, and once the final Crypto Key is obtained, you dump the program, which dumps the Pi Index file too.  Now you send your friend the key via email.  He opens the software on the other end, which loads Pi into memory again, sure, but the point is that the internet never saw the data.  The data did not choke up the middleground.  No data had to actually travel over the servers, no file analysis was done on it by the NSA, and a host of other cool benefits.  

Also, as I mentioned before, I am not going to be attacking 100GB files directly.  We can re-use the same 1 GB or 2GB Pi index and cut the program into chunks and direct each chunk through the same pipeline, so that the final file looks like this:

[OPENFILE]
[filesize=100000000000000_&_Name="100GBofStuffForWhomever.avi"]
[MChunks=50_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[2,w, xxxxxx, y, zzzz]
[3,w, xxxxxx, y, zzzz]
[4,w, xxxxxx, y, zzzz]
[5,w, xxxxxx, y, zzzz]
[6,w, xxxxxx, y, zzzz]
[7,w, xxxxxx, y, zzzz]
[8,w, xxxxxx, y, zzzz]
... etc
[CLOSEFILE]

which then get stitched back together at decompression time onto the hard drive in the original form.

Forget about 8/16/32/64 bits games or programs because those are sizes of the instructions and registers computers use to execute instructions and it has nothing to do with a topic in hand. I gave you 64 bit just to illustrate how many possible combinations a file of 8 bytes can contains I could of given you a file of size 72 bits or 1 kilobyte as an example. Do you know how many possible files could be created with the size of 1Kb exactly? 2^8192 do you know how big is this number?. How are you going to represent all those files with and index which is less than 1Kb in size? You are right that so many files do not even exist, but how would you know once you use your "method" to "decompress" this file that the file you have found is the right one?

What would you call being able to backup all of your movies in total Blueray or HD quality to your USB thumbdrive and being able to carry them in your pocket for use at a friend's house anytime you wanted?  That would be coolness.

Well, what would you call being able, as a CEO of a company for example, to add a whole bunch of important files into a rar file that you need for a meeting, but which are top secret and there are spies everywhere trying to get your data, but you put it in a container in Nature (more aptly put, you pull out of Nature a name for that file which is its thumbprint) and take that code in your own mind with you to the meeting, and then generate that data on the target computer without the use of any USB drives, Cloud Caches, internal servers, etc...  I would call that better security than most, because chances are, people would not even know you are carrying anything with you in the first place.  And you can leave your sensitive laptop and truecrypt drives behind.

What would you call being able to send a huge videogame over the internet in moments and install itself without the need to host huge expensive servers?  I would call that a huge windfall for businesses.  No more huge server farms to send huge files or do downloads or updates.  Files are compressed under this theory and sent in 4k packets.  You could use a standard dial up modem and still achieve downloads of what would amount to Gigabytes in mere moments on the client end of things. 

It goes on and on. 

Come on, are you saying this isn't a valid idea in the first place?  Now I am beginning to question if you are for real.  Maybe you are just heckling me for the fun of it.  In that case, let this thread die and go back to your normal lives.  Jeesh.  People are unbelievable.

"All truth passes through three stages. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as being self-evident.

Arthur Schopenhauer, German philosopher (1788 – 1860)"

Kinda reminds me of the feedback I got explaining a decentralized currency that would eventually take down central banks (pre-Bitcoin), but lacked any technical abilities to make it a reality (thank God for Bitcoin).

Don't let them get you down.

If you belief in your idea, consider this approach... whats the absolute minimum viable proof of concept?  What would it take to create it?  Then find a way to get proof of concept working.

No, the key is that size.
You cannot (on average across all files of that size) compress a 1MB file to an index smaller than 1MB.

"The fact that some geniuses were laughed at does not imply that all who are laughed at are geniuses. They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown."
Carl Sagan

You tried to create an "Magic" algorithm to compress/encrypt data at the same time. How one could do it without any knowledge of the informational theory or even ability to program or even knowing what is out there already. I call it blissful ignorance and yes, your idea to compress huge amount of data into a simple reference is stupid. I did call your idea stupid because it is based on your ignorance and luck of the knowledge and your refusal to educate yourself before trying to find investors for your pipe dream.

Yes, that is what we have been saying right from the beginning.
Your claim that you have a compression method that will shrink all files is simply not possible.
We've shown you the maths proving this.
The fact that it would be really cool if it was true doesn't just make it true.

At long last the penny has finally dropped!

Oh well, another million dollar idea collapses. Back to the drawing board....

Maybe for you, but not for me.  I've spent years on this, and about 1 year testing it by hand.  I had a computer-programmer friend helping me for a time, but he had some work issues and couldn't stay with it for long, so I got another friend to help me.  I taught him the rules and got him to send me Crypto Codes of 3 bytes, or 6 bytes, at a time to me following my rules.  It would take all day to do just one of them by hand, but eventually I would return the answer and my friend was totally amazed when I got each one right.  There was ever only one possible answer to be found.

The math itself doesn't lend any credence to the idea that just because there are 4.29 billion combinations, all 4.29 billion combinations are orderly arrangements that would even return a working file to begin with.  Nature probably has some kind of limits built into it inherently, whereby only 10-20% of all the possibilities will ever be arranged into an orderly thing which, on a computer medium, would be called a working file.  The chances that the Pi Index would actually require every Pi index to be the end of a working file seems ludicrous, Nature would then be shown not to be random or chaos filled at all, but totally derivitive and pre-programmed, where even static or noise would mean something if you knew how to organize it.  We are getting into philosophy here, and I don't wish to do that. 

Bottom line, none of you really understand my theory well enough to be able to discredit it so quickly, you have no right.  You are not God.  You may be smart, but you also don't see all ends, either, as history has shown among scientists most create something that goes on to become the rope for the world to hang from.  I guess by trying to create this technique, I am jumping in the scientific boat, but still ...   By the way, I'm not blaming anyone for not understanding my theory, I haven't released all the rules or explained everything, how could you?  You are only guessing at this point.  Of you all, at least BurtW seems to understand by the way he asks questions, I believe he is sincere and trying to understand, and even offer some good feedback.  Thanks to BurtW for making my day, even when you disagree, you do so with respect in your tone.  I appreciate that.

No, I am a mathematician.
We don't need to understand exactly how your 'compression' algorithm is meant to work.
You cannot compress gigabyte files into a couple of dozen characters. Can't be done.
N digits of alphanumeric index can only index 62^N maximum possible unique files. Fact.

Your trying to fit a pint of milk into a half-pint glass. Can't be done.

I have simple questions for you.
- Can you compress ANY file (to 0.2% of original size or whatever number lower than 100%)?
- How fast is your compression an how fast is decompression?
- How much additionall data do you need for decompression? (For example, you compress Blue-Ray movie to 100 000 bytes, you sned me those 100 000 bytes, but I need some other data. At least your decompression program. So how much additionall data [not counting operating system like Linux or Windows] do I need?)
- How much can you compress 1000 bytes file (text, audio, video, random data)?

Is it something like this? (https://github.com/philipl/pifs)

Maybe that as the base code + "Variable run length search and lookup!" (see the bottom of the description under future enhancements)

Now this idea does have some things in common with mpeg:

1) Encoding is a long laborious tedious process.  And, the more resources you spend encoding (in this case time and money spent doing the variable run length searches) the smaller the compressed file.  The encoding could be done on "very big iron" in order to compress high value content like the latest Adam Sandler POS movie.

2) Theoretically the decode could be fast / real time and most importantly it can be done in parallel.  Knowing the next N metadata descriptors you can immediatly calculate all of them in parallel.  And a hardware decoder is theoretically possible.

Now to see if it is practical we can take a search space S = a number of digits of pi, and calculate the average run length possible within this search space given random input data.

Since each metadata descriptor would have a starting bit or byte location and a number of bits or bytes the average metadata descriptor size must be smaller than the average random sequence found in the search space.

One other thing to note is that the size of the search space S only affects encoder search time and cost so it can be made very large.  It does not affect the decoder cost and only affects the metadata size in that every time you double S you have to add one more bit to the location parameter in the metadata descriptor.

Hmmmmm....

 

What an idea..The maths dont add up though.

ROFL

you made my day

thank you sir !

From a cursory reading I'd say the cost of energy required to 'brute force' a solution (ie recreate the file) just from a cryptographic hash of said file is orders of magnitude higher than the raw transmission cost.

Is this for real, or a is it a joke?  Did someone just make this site to mock me, or is this for real? 

The description used mirrors my idea exactly .... perhaps someone has already discovered what I am trying to get done.  If this is so, what a pity.  My dream goes to another. 

My theory's encoding speed is the opposite of the industry, then, in this case.  The encoding time is real-time encoding.  Meaning you can record video straight into the formula and all that gets saved to disc is a 4 kilobyte file.  In this case, it's the decoding that super slow, at least I think it would be. Without testing, I couldn't be certain.

During my research, I found that the average space needed to encode 8 bits (1 byte) is 100 to 150 Index points into Pi per byte.  Thus, if you have a 1024 K file, you would need 12,240 index points to reveal the thumbprint of that data and obtain the Crypto Code.   If my math is right (and it might not be, so please check me, I'm not good at math) it would take 12.24 BILLION index points into Pi to store 1 GB of data.  I don't want to go that far into Pi because then just trying to find the timeline in all that branching data could take lifetimes, so I want to break up the file into multiple Crypto Keys, spaced evenly.  SO lets say 250 Megabytes or 500 Megabytes .... Or Even 100 Megabytes if that is faster. Since each 100 megabytes of data would be represented by a single line, adding more lines or more chunks wouldn't really cost anything:

100 Megabyte File Stuffed In 100 Megabyte Splits
[OPENFILE]
[filesize=100000000_&_Name="100MBofStuffForWhomever.zip"]
[MChunks=1_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[CLOSEFILE]

400 Megabyte File Stuffed In 100 Megabyte Splits
[OPENFILE]
[filesize=400000000_&_Name="400MBofStuffForWhomever.zip"]
[MChunks=4_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[2,w, xxxxxx, y, zzzz]
[3,w, xxxxxx, y, zzzz]
[4,w, xxxxxx, y, zzzz]
[CLOSEFILE]

each using the same 1 GB index key from Pi to be encoded through.  The data is then searched backwards from that final Index point in Pi to the decimal point (or start of Pi) to find the one unique timeline that fits all the criteria.  Since this approach uses a way of tracking how many 1s are in the data, what the initial conditions are at the start of Pi, it can use this information to search backwards from the ending point of each Mega Chunk of data to find the unique timeline.  Once a timeline is captured, you now have the data that was originally encoded by measuring a set of changes you made while travelling forward in Pi.  There is thus only one unique route taken, and thus only one unique data set that can be found for the entire sequence.  Even if you arrive at the same number in Pi as and ending point, the individual changes made inside the route can also be tracked to ensure uniqueness.  Even if ten different files land on the same ending point in Pi, that does not mean the theory is broken, since its the signature of the timeline itself and how it fits like a key into a very exact and precise lock.  When this is found, the program can draw out the data by analyzing and reconstituting the 0s and 1s precisely as they were added in during the path taken.

I realize this doesn't seem to make sense, because language cannot express what I have seen, or at least MY language cannot express what I have seen to be true and demonstrated on paper.  I'm sure many people have had ideas they have failed to be able to adequately communicate but which they could see themselves clearly, like Leonardo Davinci.

So this is where you copied the idea from.

Asking for investors, yet was open source all along.

Code is fully functional BUT it is a joke (published on 1 april)

Anyone who knows a little math can tell compressed files will be BIGGER then the original.

Someone else told you that:


This is also VERY VERY SLOW

You are not a teacher

When I was about 13-14 years old, I had a similar (yet simpler) idea of "compressing" files by running loooong seeded random number sequence and waiting for sequence-to-be-compressed to appear.
Took me some time to realize why it can not work. (Well, It can work, but the indexes pointing into sequence of random numbers [and same applies for pi] would be bigger than the data I wand to compress.)

The basic problem is lack of imagination. Every file appears in Pi, that is right. But the first time, first decimal place, some specific 1024 bytes file appears in Pi can be so high number that you can not write it by 2500 decimal places (that means that you can not write it by even with 1024 base256 numerals [= bytes]).

You can try to "compress" 3 byte NUMBER sequences (000-999) by waiting for them to appear in Pi. For some of them, you will wait longer than 10^3 = 1000 decimal places to appear.

This is a must, because number of possible sequences to be compressed is always bigger or equal the number of possible indexes/decimal places/descriptor/metadata pointers.


I am sorry to disappoint you though :-/

http://blog.dave.io/2013/03/fs-a-filesystem-capable-of-100-compression/

according the above-linked description, pifs works by storing an index for every byte.  So you usually end up with the indexes being larger than the compressed data.  I get that.  Also, looking for a full file (byte-for-byte) in pi is a computationally expensive proposition.

Question: What if instead we use indexes to fixed (or variable) length sequences instead?  But only when the sequence length is greater than the index size.   And pre-calc pi up to some limit so we can just lookup index into pi given a sequence.  If not found, just store the input bytes instead.  When compressing and decompressing, we could also check for rotated values of our input string.  eg:  1,2,3,5 could also match 2,3,4,6 or 3,4,5,7 in our lookup table -- so long as we store a rotation offset.

waiting for all the mathematians to explain why this is unworkable.    ;-)

To try to drive this home, lets use your own example.
You have encoded three 7 bit numbers (actually one of them is only 6 bits, but I'll assume that was a typo.
A 7 bit number can represent the values 0-127.
A 6 bit number can represent the values 0-63. And so on.
When encoding your three 7 bit values, for two of them your resulting index is also a 7 bit value.
So your index needs just as much space to be stored as the original data did.
This is what we've been trying to tell you, and your own example has shown it to be true.

And just to quote from that:

We aren't just being mean to you, it really doesn't work.

 I do believe this can be done , maybe not by op , maybe not anytime soon , but one day this idea will be done . 50 years ago whe had no internet , we created our own made up currency bitcoins on this internet because we were sick of our countries currency and that bitcoin is over 100$ each now , if anything bitcoin tought me it is everything is possible . Not now maybe , but one day.

You don't know if index will be smaller or larger BEFORE calculating it.
In most cases (more than 50%) it will be larger
In some cases it will be a little smaller
You also have to store an additional information for every block: am I saving an index or the sequence?

Compressed file will be bigger than 50% of the original file (perhaps bigger then the original file because of additional informations)


So the exe becomes huge and needs a lot of RAM


You reduce exe size but have even more informations to store in the compressed file (making it bigger) and need more computational power (pre-calc becomes quite useless)

Probably you are wrong.

Internet was created by men
bitcoins price is decided by men

math was not created by men, nowadays we have no way to change it

I think in the future 1+1 will always be 2 (can't be absolutely sure about this)

But this was only an example of how the encoder works, not its compression value.  I told you before, this is for larger than 500k files.  You don't seem to understand, now matter how many times I say the same thing different ways, that this one code represents whatever size file you are working with.  If the file is 1024K or 1024 MB, it's still the same one crypto key.  Whether it's 1 MB or 100MB or 1000MB, it's just one 4k file.  You aren't listening.  Your only job appears to be to find ways to break my theory using small data sets of 3 bytes, when I clearly said many times this won't be used for data that small.  

Let's say I encode Apple's iCloud Installer, which is just under 50 MB in size (according to Google).  For every 1 byte of data, I need to index 100 - 150 indexes into Pi.  I am not converting anything or recording anything to do this movement.  I am simply moving forward according to some rules I figured out for creating a unique pathway through Pi.  So I would need (100*50,000) or 500,000 indexes into Pi at least.  Let's say that the last bit of data I find is at index location 501,500 into Pi.  Well, here is my crypto key:

[0.501500.8.5250]  I didn't have to record any other data that than.  Now that's placed into the 4K file that is used to tell the software how to reconstitute the data.  This is ALL the data there is in my finished file, plain and simple.  There is no hashing chunks of data as you describe, nothing like that.  I've created a path through Pi like taking footsteps on certain numbers.  The footsteps taken are irrelevant, what is relevant are the changes between those steps.  For 0s, we hop from the first 4 in Pi to every other 4 in Pi only.  But if we encounter a 1 in our binary data, then the Hunt Value increments +1, so now we are only hopping on every 5 in Pi.  This is what keeps the record of our original bits.  All the other numbers in Pi are useless as we are encoding.  Here is an example:   001      We would be looking for the first 4 in Pi then the 2nd 4 in Pi and now we must increment +1 and hop to the next 5 in Pi.  We keep hopping on 5s as long as our data is 0s, but if we encounter another 1, we increment and begin hopping along 6s.  In this way, our pathway is totally unique to the data we are feeding into it, and thus our arrival point (end point) can let us figure out the path taken to reach itself by knowing how many 1s were used and then attempting to solve backwards to the decimal point using the precise number of steps it took to encode it, the original file size recorded in bits.

Also, I'm not sure I should be saying 8 bits per byte, but my friend taught me 7 bits per byte when working with Ascii Binary, was I misinformed on that?  Remember, the theory works by looking at the data in a file as characters in a book, in Ascii format, and thus will need to encode precisely the same number of bits for every character.  But if you look at a hex and decimal and binary converter online, if you type in just one letter, you only get 3 bits or 4 bits or 2 bits ... some erratic bit size.  Every character must have the same bit size, so I want to translate the data into Ascii Binary.

Listen, if I sat down with a programmer who was able to listen to what I'm saying and not throw up objections that have no relevance to my method, and we could sit and work on this until it fits my theory exactly, then I know it would work.  You are right, I cannot do it myself, but I have been very clear about that from the very first post.  I have asked for help.  But it could be done soon.  Very very soon.  A few days for the encoding portion, it's just a file lines really.  The decoding portion would take a lot of brain work, research, testing, modification, etc ...  and we'd have to make our choices about how to resolve the solution of this software by seeing the results of those tests.  If I had the money to hire a coder myself, I would not have even come here at all.  Again, not asking for money now, but I am asking for someone to just TRY to do this with me.  How much time could it take to prove me wrong?  Maybe less than the time it's taking you to actually type out these responses to belittle me so you can go on believing everything is impossible.

Stop trying to push this off to ... someday, when now would be just as good.  Another person besides myself, his name is Philip Langdale, a programmer, has already created something like my idea here:  https://github.com/philipl/pifs

It is a method (like I've been telling all of you) to push data into Pi using a very hard mathematical solution.  It barely works because the encoding is just too slow.  But he has already done this!   You can download the app yourself and give it a try.  He is encrypting data into Pi.  But he's doing it the wrong way in my opinion.  But if you read his page, you will see that he has rational support behind that idea that every known file in the universe can fit inside of Pi.  Go read his arguments yourself, so you can stop beating me up for things you don't understand, despite being geniuses no doubt.  I want to work with you, I don't want to fight with you.  Thanks for your time.

There are little more than 1.415461031044954789 x 1E9864 possible 4k files. These are ALL possible results of your compression. Although this number is extremely high, it is much lower than number of possible 1MB or 100MB or 1000MB files.
So how can you make a correspondence (1 to 1 connection) between a set of files and much larger set of files without any code repeating (correspondig to several different decompressed files) ??

You do realize that this whole "Pi FileSystem" was a practical joke, right?

Yes, it works, in theory, for 0.000000001% (prolly much less) of all possible input files. But it's way, way, WAAAAY too slow for practical usage AND it doesn't work for the remaining 99.999999999% (prolly much more) input files.

And "too slow" in this context is not just a matter of requiring faster processors, it's "too slow" as in "takes longer than the age of the universe to process".

You don't seem to understand when I try to explain very clearly why your process will not work.
You've even seen a joke page created by someone else with broadly the same process, and another page explaining why it will not work.


a) 50,000 is 50KB, not 50 bytes. 50MB is 50*1024*1024, not 50*1000.

b) And what if you don't find the last bit of data until index location 501,500,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000...(repeat up until 50MB)?

You entire process only works if you assume you can find all possible index locations in less space that the original file took.
You cannot do this.


And what people keep telling you, but you refuse to accept, is that on average, the index required to store the final position will be as large as the file you are trying to store.

This. You're compressing multiple different input files to the same 4K crypto key. When decompressing a 4K crypto key, how do you determine the result, since there are multiple (or actually: infinite!) possible outcomes.

You seem to be missing this critical point, B(asic)Miner. There are infinitely more files larger than 4K, than there are 4K crypto keys.

Yes. Every known file in the universe is somewhere inside of Pi. (As a matter of fact, even every finite-length unknown file is located inside Pi. That means every wikipedia article that will be published, description of every invention that will be invented on Earth or elsewhere, every novel, short-story or book that was, is or will be written. Incuding all variants of ending, all misspellings, etc.)

But no matter how you traverse the Pi you always need a starting point, starting index, first decimal place to begin with. (Or maybe you need last index, last step of your path for recreating the path from finish to the start.) Obviously simplest way is to just proceed sequentially, start eg. with index 12345678 and proceed to 12345679, 12345680, 12345681, ... But even if you traverse or backtrack Pi some different algorithmically predefined way, you need that starting point.

Number of starting points can not be smaller than the number of possible compressible files.

This isn't even certain, it is not known whether Pi is a normal number (http://en.wikipedia.org/wiki/Normal_number) or not.

So, even the algorithm that B(asic)Miner claims to have written, is located somewhere in Pi already. Therefore it must exist. Or, ehh... wait :)

I'm not sure I got it right...

You store in your 4k file final position and path length... how can you say the path is unique?

Take this part of pi:
3.141592653589793238462643383279502884197169399375

These two sequences have same lenght and end on final 5:
00010
00001

I happen to have done so recently:

Ah, see, conjectured. This is NOT AT ALL proven to be true whatsoever. To this very day, no mathematical proof exists that states whether or not Pi is normal.

(and even if it is, the idea is still flawed for obvious reasons already described above)

B(asic)Miner, how about this: wanna bet? Walk your talk? Put your money where your mouth is? After all, money talks and bullshit walks :)

I give you a few 1 MB files (1048576 bytes). If you can compress them to 950 KB (972800 bytes) or less, I pay you 100 BTC. If you are unable to pull this off within a reasonable time limit, let's say one or two months from now, you pay 1 BTC to any charity of your choice (must post tx here ;))

Deal?

 I thought this was possible? Aren't we able to rip games and all? I dont get it why cant he compress 1mb into 950kb ?

 This is not 'of course it can be done' message , it is more like 'I really dont know if we are able to do this or not , I tought we were able to do it but I might be wrong' type of message

True. I forgot about unproven normality. So lets rephrase this and say that it is instead in sequence of truly random numbers. (Well, actually you can only prove, that every finite-lenght sequence is encountered [i random number sequence] with probability which converges to 1.)

Yes, the B(asic)Miner's algorithm is there. That does not mean it does work :-]. All algorithms exists there even non-functiong ones.


But there is interesting question... How many different computer files are there? I mean... the number of possible files is easily calculated given length limit. But how many different files truly exist up today?
So imagine central database with one indexed copy of every file. (Every time any new file is created or existing file is somehow changed in the slightest, not yet encountered way, it is stored into central database under new index.) Such set of indexes would be manageable. However, the process of upgrading the database and downloading files based on index would be not.

Another interesting question: Is it possible to create "supercompression" as non-deterministic probabilistic algorithm? That means one compressed file can produce multiple outputs but the probability of correct one is >99.999997% for long enough processing.

Entropy is the answer! :-]

Your right, but finding the position where it is is the problem, and the position index will be likely even bigger (in Bytes) than the data you get (and dont start the loop - we will encode this way the positin as well  :D)

Oh, he'll probably be able to compress some 1MB files to 950KB. But not the ones I'm supplying :)

See, compression is all about information density. Quite some everyday files of 1MB contain much less than 1MB of actual information. A text file, for example, or an image with lots of 'empty' spaces, will not have a very high information density. Meaning they use 1MB of disk space to store a lower amount of actual data.

But if an 1MB file actually holds (close to) 1MB of information, there's no way to represent that same information in significantly less disk space. Brilliant compression algorithm or crypto key magic or not. And to get the proper persective: the vast majority (like 99.9999%) of all 1MB files actually do hold pretty close to 1MB of information.

Either way, I have enough faith in these principles to set a 100 BTC bounty to be proven otherwise. Hope to hear from B(asic)Miner soon. :)

 Well , if I had 1 bitcoin to bet , I would try to get you on that bet but I dont. I would tought take you on , if I can compress one 1mb file you send me into 950kb you will pay me 1 bitcoin , if I fail I will do something of your choice that doesnt involve any sorts of money (since I cope with poverty after some bad investments :/)

Sure, I'll be happy to do that.

Let's be clear about the definition of 'compress' though. Either of these conditions is fine with me:

1) I provide the 1MB file in advance. Then you provide a smaller ('compressed') file in return, plus some decompression program or script or anything, which can recreate the original 1MB file. The size of the compressed file and your program or script together, may not exceed 950K.

or

2) I provide the 1MB file in advance, in encrypted form. You provide two programs or scripts or whatever. The first is able to take an input file and reduce it to 950K or less. The second is able to take the compressed file and restore the original 1MB file. These two programs or scripts can be as large as you want. May also be just one program that does both things. Then I provide the password to get access to the 1MB file I supplied earlier.

These two approaches are to avoid nonsense solutions where the data is actually just moved into the decompression program. E.g. simply cutting off the last 50K of my file and storing that in the 'decompression' program.

In both cases, things must still work if I rename the program and/or compressed file. This is to avoid nonsense solutions where data is stored in the filename.

Let me know if you want some 1MB files to test :)

1 BTC for every success. No action (other than the effort you put into compressing them) required in return if you fail.

 Let me put this out there , I have no technological or math skills , I was merely joking and would probably put whatever you send me int o a .rar file or search google for some methods and if i fail i fail :D

 So basically , we could already see where this is going , I wont win , but I will try just for the heck of it :D

 I do not understand your propositions , both of them . Am I allow to try to .rar whatever you send me? How much time do I have? :D Its like asking for a 12 year old to score a touchdown at an NFL game , there might be a sheer luck where a 12 year old may run 1 yard and score :D , very low chance but it worths the try for 1 bitcoin since there is nothing I could lose :D

 Btw that means : sure mail me some stuff and I will try to compress them , you can find my mail on my profile.

Can you compress compressed file? Can you RAR .rar file? :-]
Even 99% compression (1% reduction of file size) working for ALL files would be impossible as you could run the file through this compression multiple times.

I find your idea described here:

https://bitcointalk.org/index.php?topic=288152.msg3120662#msg3120662

very interesting from a purely mathematical, theoretical point of view.

Could a very large out of band reproducible random data set (like pi) be used in this way?

Can we uniquely encode a binary input stream as a very small set of metadata in this way?

Correct me if I am wrong but your theory can be summarized as follows:

The metadata (your very small file description) consists of an end point in the number pi and number of hops it took you to reach that end point into the number pi.

Your encoding is just finding the end point and counting the number of hops.

Your decoding is finding the one unique path that ends up at that specified endpoint using that specific number of hops.

First, I must say that I do not think the general purpose decoder for anything other than trivial examples is possible by current computational means.  As an exercise it might be fun to code up a simple encoder and decoder that could be used to try this out on trivial (up to a hundred bits or so) examples.  Who knows, something interesting might come out of the exercise of trying to find an efficient way to find the path (or paths see the next point below).

Second, you would need to prove that given an endpoint and a number of hops there is only one solution (path).  I have not put too much thought into this but this looks to be a very hard thing to prove.  However, the good/bad news is that the opposite assertion:  given an endpoint and a number of hops there is not always one unique path looks to be very easy to prove.  We just need one single example where we find more than one path to an endpoint that takes the same number of hops.  The simple program described above could be used to prove this.  All you have to do is pick random input data sets, calculate the encoding and then run the decoder to find all possible solutions.  If there is more than one solution in a few cases (my expectation) then we have proved the negative assertion.

I for one am very glad you thought this up and have spent so much time and effort on it and want to personally thank you for bringing it to my attention.  It is fun to dream up new things and share them.  You could have saved yourself a lot of grief and flames by just publishing your idea in your original post but I know from personal experience that is very hard to do when you think you have really got something new and valuable.

Oh God, please just give OP a bitch-slap!

Everyone:

I also know how much fun it can be to bait, flame and troll some of the losers and scammers who post here.  I think this guy is simply trying to find out if his idea will work.  I know I could be wrong and he may just be having fun or trying to pull a scam - it is so hard to tell sometimes but so far he looks genuine to me.

Carry on...

Although I highly appreciate creativity and thinking out of the box, this whole idea is fundamentally flawed, as could have been proven in advance, regardless of the exact implementation details.

Not because of 'stubborn thinking' or refusing to look beyond known methods, but simply because it is mathematically impossible to create a compression (or 'encoding' or whatever you wanna call it) algorithm that reduces any file to less than its own size. No matter if you use Pi, or black magic, or voodoo, or whatever.

Let's repeat the same argument one more time: there are N possible files of 1MB. There are M possible 4K crypto keys. N is much, much, MUCH larger than M. So, by definition, the vast majority of 1MB files (that is, way over 99.9999999999999999999999%) can NOT be represented by a 4K crypto key. Period.

Even if you would create a HUGE (de)compression program that contains a vast library of ALL possible 1MB files, and when compressing a file it just stores some index to specify the exact 1MB file you need, it's still impossible. There's much more possible input files, than the number of unique index keys (or 'compressed files') you can output.

What riddles me, is why people who are obviously not stupid, keep believing in fairy tales and ignoring plain simple logic.

That is why I called it a "very large out of band reproducible random data set (like pi)"

That is why it is an encoding/decoding system, not a compression system.

Very large amounts of information can be encoded in very small metadata:  ISBN numbers, URLs, etc.

He is not talking about data compression at all.  He just does/did not know the proper terminology is all.

Ok, to go the extra mile, I have written a toy program which does what I think you are describing:
- Set the first Pi search digit to 4
- Start at the beginning of the digits of Pi, and the beginning of an input file
- Repeatedly:
  - Take the next byte of the input file and convert it into binary
  - For each binary bit:
    - if it is is 1 increment the Pi search digit, wrapping round to 0 as necessary
    - find the next digit of Pi which matches the search digit
- Print the resulting index digit of Pi where the last binary bit of the last byte of the input file was found.

The program is extremely slow, so I'm not going to run it on a huge file.
For the first just over 1.5KB of its own source file, the program produces:

After byte 1610, pi position is 130339
After byte 1611, pi position is 130407
After byte 1612, pi position is 130468
After byte 1613, pi position is 130531
After byte 1614, pi position is 130666
After byte 1615, pi position is 130747
After byte 1616, pi position is 130857
After byte 1617, pi position is 130915
After byte 1618, pi position is 130995

I hope you can see that your 50MB file is never going to have a final index position of 500,000, if I have already reached 131,000 with only 1.5KB.

[Note that you haven't demonstrated any decompression routine, and this 'compression' routine is not proved to be reversible, so that two input files may give the same index value, which means that decompression is not possible.]

OOPS, reading back through the previous posts I see this has already been shown here:

https://bitcointalk.org/index.php?topic=288152.msg3120822#msg3120822

Oh well, might as well post my work here anyway....

-----------------------------------------------------------------

Thanks for writing the program!  Cool, I was just about to do the same.

Here is a quick example showing that the paths are not unique for the idea as described.  We may not have the full description and there may be more to it than we have been told but using the start at 4, jump to next 4 as long as the input is zero and increment when the input is one idea:

The following two sequences must have different paths:

00001000
00010000

Assume what if we are in an area of pi with the following sequence of digits:

   4 ... 4 ... 4 ... 5 ... 4 ... 5 ... 5 ... 5 ... 5  (where ... means digits that do not matter in this example)

If my understanding of the idea is correct both inputs land at the same end location.

00001000 would map to 4 4 4 4 5 5 5 5
00010000 would map to 4 4 4 5 5 5 5 5

but with the totally possible distribution given we end up at the same ending location.

Is our understanding of your idea complete and correct or is there more to it than described so far?

I changed my program to use a precalculated file of Pi to 1000000 digits, rather than compute the digits on the fly.

When I apply it to the same 1000000 digits file, it runs out of Pi digits at only 12500 bytes of the input file (out of 1000000 bytes total).

'Compression' stats at that point:
After byte 12505, pi position is 999428
After byte 12506, pi position is 999525
After byte 12507, pi position is 999561
After byte 12508, pi position is 999628
After byte 12509, pi position is 999729
After byte 12510, pi position is 999795
After byte 12511, pi position is 999856
After byte 12512, pi position is 999950

Makes no difference. There is no way to 'cheat' the problem at hand. On average, the index will be at least as large as the actual data itself, and very likely (in 99.999999% cases) even larger.

Besides some exceptional (very rare) 'lucky' numbers, the vast majority will require more data to store the index.

Consider this: even if it would be possible to compress (or 'encode' or 'represent' or 'index') just a tiny bit off any file, you could repeat the same process over and over, and eventually reducing every file to a few or even one bit. I think you'll intuitively feel that's not gonna work.

Pi is even a transcendental number (http://en.wikipedia.org/wiki/Transcendental_number) (as opposed to e.g. √2 which is also irrational, but algebraic (http://en.wikipedia.org/wiki/Algebraic_number)).

B(asic)Miner questions and specific answers for you:

Where did you get this 100-150 number?  

We would need to see all the rules in order to help you figure out if this idea is theoretically sound.

As pointed out previously a 50 megabyte file contains 50*1024*1024 = 52,428,800 bytes.  So using your number of 100 digits /byte you would need at least 5,242,880,000 digits of pi.

Please stop using this term.  It has a meaning and this is not it.  Use "here is my metadata"

Is this the full description?

I don't think this is true.

Even if the path was unique, which I don't think it is, this process sounds very very hard, possibly impossibly hard.

A byte is always 8 bits by definition.  How that byte and those 8 bits are used to encode data varies.  (Simple) ASCII maps all the various printed symbols into these 8 bits.  8 bits per character.

Don't worry too much about these details.  Just describe the idea itself as well as you can and let the geeks and engineers worry about the details - like whether it is possible or not.

Not true at all.  If the idea worked, and it does not, but if it did the idea would produce just three small numbers, something like this:


This is far less than the input of 50 megabytes.

 I am still waiting for 1mb file mail :D Send it to me , I will try to compress it , what can I lose but time :D

Now the next question is if this is possible, can a consumer computer do a compression within few seconds if not minutes for average file sizes?

The biggest question is not whether a computer can do the "compression" but whether it is possible to reconstruct the file.

Here are the chains of digit positions for just 4 bits:

Pi:    314159265358979323846264338327950288419716939937510
0000     4                4   4            4
0001     4                4   4       5
0010     4                4           5                5
0011     4                4           5         6
0100     4 5   5 5
0101     4 5   5           6
0110     4 5  6            6
0111     4 5  6     7
1000       5   5 5                    5
1001       5   5 5         6
1010       5   5           6 6
1011       5   5           6        7
1100       5  6            6 6
1101       5  6            6        7
1110       5  6     7               7
1111       5  6     7    8

As you can see, the bit combinations 0101, 0110 and 1001 all lead to the same ending digit position. The same is true for 0001/1000, 1010/1100 and 1011/1101/1110. This means that the algorithm's output of a file starting with one of the bit patterns in such a group is indistinguishable from the output if the file had started with one of the other bit patterns in the group.

It follows that it is impossible to uniquely decompress the output.

Don't take it personally, but your scheme is not usable.

Onkel Paul

I am not trying to get the system, as described, to "work" but out of mathematical curiosity I wonder if there is a starting search digit that would create unique ending positions for all 16 of the 4 bit combinations.  We would have to try starting at 0, then 1, then 2, ... up to 9.

If there is one of more starting locations that do in fact lead to unique ending locations for all 16 of the four bit combinations then we could see if any of those could be extended to 8 bits.

If there is even one starting digit that creates a unique map of the 4 bit values or the 8 bit values then it could be used as an esoteric code mapping.

This could also be tried on all 16 starting locations of a hexidecimal representation of pi.

Pretty simple program.  If I get time I might try it for fun.

If anyone else is still reading at this point, and wants to check my working, I think I've demonstrated clearly that this is not a reversible compression scheme.

The two letter word "Pi" gives an index of 148.
The following 156 two letter words also give the same index:
(This is restricted to just printable characters, there could be far more matches using the full 256 combinations for each byte)

String: 0i (0011000001101001) Index: 148
String: 0j (0011000001101010) Index: 148
String: 0l (0011000001101100) Index: 148
String: 1I (0011000101001001) Index: 148
String: 1J (0011000101001010) Index: 148
String: 1L (0011000101001100) Index: 148
String: 8I (0011100001001001) Index: 148
String: 8J (0011100001001010) Index: 148
String: 8L (0011100001001100) Index: 148
String: 9A (0011100101000001) Index: 148
String: 9B (0011100101000010) Index: 148
String: 9D (0011100101000100) Index: 148
String: :A (0011101001000001) Index: 148
String: :B (0011101001000010) Index: 148
String: :D (0011101001000100) Index: 148
String: <A (0011110001000001) Index: 148
String: <B (0011110001000010) Index: 148
String: <D (0011110001000100) Index: 148
String: @; (0100000000111011) Index: 148
String: @= (0100000000111101) Index: 148
String: @> (0100000000111110) Index: 148
String: @[ (0100000001011011) Index: 148
String: @] (0100000001011101) Index: 148
String: @^ (0100000001011110) Index: 148
String: @k (0100000001101011) Index: 148
String: @m (0100000001101101) Index: 148
String: @n (0100000001101110) Index: 148
String: @y (0100000001111001) Index: 148
String: A9 (0100000100111001) Index: 148
String: A: (0100000100111010) Index: 148
String: A< (0100000100111100) Index: 148
String: AK (0100000101001011) Index: 148
String: AM (0100000101001101) Index: 148
String: AN (0100000101001110) Index: 148
String: AY (0100000101011001) Index: 148
String: AZ (0100000101011010) Index: 148
String: A\ (0100000101011100) Index: 148
String: Ai (0100000101101001) Index: 148
String: Aj (0100000101101010) Index: 148
String: Al (0100000101101100) Index: 148
String: B9 (0100001000111001) Index: 148
String: B: (0100001000111010) Index: 148
String: B< (0100001000111100) Index: 148
String: BK (0100001001001011) Index: 148
String: BM (0100001001001101) Index: 148
String: BN (0100001001001110) Index: 148
String: BY (0100001001011001) Index: 148
String: BZ (0100001001011010) Index: 148
String: B\ (0100001001011100) Index: 148
String: Bi (0100001001101001) Index: 148
String: Bj (0100001001101010) Index: 148
String: Bl (0100001001101100) Index: 148
String: CI (0100001101001001) Index: 148
String: CJ (0100001101001010) Index: 148
String: CL (0100001101001100) Index: 148
String: D9 (0100010000111001) Index: 148
String: D: (0100010000111010) Index: 148
String: D< (0100010000111100) Index: 148
String: DK (0100010001001011) Index: 148
String: DM (0100010001001101) Index: 148
String: DN (0100010001001110) Index: 148
String: DY (0100010001011001) Index: 148
String: DZ (0100010001011010) Index: 148
String: D\ (0100010001011100) Index: 148
String: Di (0100010001101001) Index: 148
String: Dj (0100010001101010) Index: 148
String: Dl (0100010001101100) Index: 148
String: EI (0100010101001001) Index: 148
String: EJ (0100010101001010) Index: 148
String: EL (0100010101001100) Index: 148
String: GA (0100011101000001) Index: 148
String: GB (0100011101000010) Index: 148
String: GD (0100011101000100) Index: 148
String: H9 (0100100000111001) Index: 148
String: H: (0100100000111010) Index: 148
String: H< (0100100000111100) Index: 148
String: HK (0100100001001011) Index: 148
String: HM (0100100001001101) Index: 148
String: HN (0100100001001110) Index: 148
String: HY (0100100001011001) Index: 148
String: HZ (0100100001011010) Index: 148
String: H\ (0100100001011100) Index: 148
String: Hi (0100100001101001) Index: 148
String: Hj (0100100001101010) Index: 148
String: Hl (0100100001101100) Index: 148
String: MA (0100110101000001) Index: 148
String: MB (0100110101000010) Index: 148
String: MD (0100110101000100) Index: 148
String: P9 (0101000000111001) Index: 148
String: P: (0101000000111010) Index: 148
String: P< (0101000000111100) Index: 148
String: PK (0101000001001011) Index: 148
String: PM (0101000001001101) Index: 148
String: PN (0101000001001110) Index: 148
String: PY (0101000001011001) Index: 148
String: PZ (0101000001011010) Index: 148
String: P\ (0101000001011100) Index: 148
String: Pi (0101000001101001) Index: 148
String: Pj (0101000001101010) Index: 148
String: Pl (0101000001101100) Index: 148
String: QI (0101000101001001) Index: 148
String: QJ (0101000101001010) Index: 148
String: QL (0101000101001100) Index: 148
String: SA (0101001101000001) Index: 148
String: SB (0101001101000010) Index: 148
String: SD (0101001101000100) Index: 148
String: UA (0101010101000001) Index: 148
String: UB (0101010101000010) Index: 148
String: UD (0101010101000100) Index: 148
String: YA (0101100101000001) Index: 148
String: YB (0101100101000010) Index: 148
String: YD (0101100101000100) Index: 148
String: ZA (0101101001000001) Index: 148
String: ZB (0101101001000010) Index: 148
String: ZD (0101101001000100) Index: 148
String: \A (0101110001000001) Index: 148
String: \B (0101110001000010) Index: 148
String: \D (0101110001000100) Index: 148
String: `9 (0110000000111001) Index: 148
String: `: (0110000000111010) Index: 148
String: `< (0110000000111100) Index: 148
String: `K (0110000001001011) Index: 148
String: `M (0110000001001101) Index: 148
String: `N (0110000001001110) Index: 148
String: `Y (0110000001011001) Index: 148
String: `Z (0110000001011010) Index: 148
String: `\ (0110000001011100) Index: 148
String: `i (0110000001101001) Index: 148
String: `j (0110000001101010) Index: 148
String: `l (0110000001101100) Index: 148
String: aI (0110000101001001) Index: 148
String: aJ (0110000101001010) Index: 148
String: aL (0110000101001100) Index: 148
String: cA (0110001101000001) Index: 148
String: cB (0110001101000010) Index: 148
String: cD (0110001101000100) Index: 148
String: eA (0110010101000001) Index: 148
String: eB (0110010101000010) Index: 148
String: eD (0110010101000100) Index: 148
String: iA (0110100101000001) Index: 148
String: iB (0110100101000010) Index: 148
String: iD (0110100101000100) Index: 148
String: jA (0110101001000001) Index: 148
String: jB (0110101001000010) Index: 148
String: jD (0110101001000100) Index: 148
String: lA (0110110001000001) Index: 148
String: lB (0110110001000010) Index: 148
String: lD (0110110001000100) Index: 148
String: qA (0111000101000001) Index: 148
String: qB (0111000101000010) Index: 148
String: qD (0111000101000100) Index: 148
String: rA (0111001001000001) Index: 148
String: rB (0111001001000010) Index: 148
String: rD (0111001001000100) Index: 148
String: tA (0111010001000001) Index: 148
String: tB (0111010001000010) Index: 148
String: tD (0111010001000100) Index: 148

In fact, even with the single character 'P', there are three other printable characters with the same index:
String: B (01000010) Index: 74
String: D (01000100) Index: 74
String: P (01010000) Index: 74
String: ` (01100000) Index: 74

I just invented a beautiful  compression software: it's called md5

It can compress files of any size to just 128 bit

Still  developing decompression tool... :)

OP is simply delusional and people keep feeding the troll. He keeps saying his yada yada yadas with no useful information. Reminds me of Herbalife; "you can be RICH like the actor in this video. You just have to buy a ton of our expensive overpriced inventory...". The focus is on how rich you can be instead of the product you'll try to sell.

Even if OP could break mathematical laws, he should look for angel investors, showing them his "discovery". Google how to patent an idea instead of wasting people's time. See how long this thread is? OP is just looking for attention or trying to scam somebody without knowledge about compression.

C'mon, don't get paranoid. This isn't any sort of scam, it is just a harmless crackpottery. Nobody's going to lose any money or health or life or anything really valuable after reading this thread. Remember that Bitcoin a currency backed by gold, commedy gold.

Here is a zip containing 3 test files of exactly 1MB (1048576 bytes) each:

http://www.mediafire.com/download/z2sniezt41c8hi0 or mega mirror (https://mega.co.nz/#!gxsUTbyD!JQmDDxapv6EVWkgkHmeeQlDUdaMsbwjcLmDbEjJOFpk)

Just to be sure you're getting the correct data, the MD5 checksums of the 3 included files are:

statistics-2b.ocx = d027872ea8e2b5b7db33cf6c2da23080
example.mp5 = e4d85efed99135c8d55ff23dc5b10c34
delta.iff = cb44279ee8c9e7a6859498f5ebd8427e

I'm curious to see if B(asic)Miner or anyone else can successfully compress any of these files to 950K or smaller. What the heck, even 990K (1013760 bytes) would already be very impressive.

Sure, go ahead. I haven't tried it but I'm pretty sure Rar won't reduce them to 950K or less (in fact, I think Rar won't even reduce 1 bit off them).

See link above for 1MB test files :)

For the 5 letter input file "Hello", I cancelled the program early on in the search for duplicate encodings, but it had already found almost 2 million five letter words which gave the same Pi index (305). There are 227k words starting with 0 which do.
Hopefully it is clear that this process does not work, as it assigns the same output value to multiple different input files, and therefore there is no way of reconstructing the original input file from just the output index.

 I am at work now , I will try and respond in about 5 hours (more or less).

 I will simply .rar these and check if that worked , if it doesnt I just lost :D

 So you can basically .rar these if you want and dont wait me for 5 hours , if it doesnt compress them the answer will be on your hands without waiting for me for 5 hours :D

ANSWERS:

A) Thanks.
B) You have just revealed that you still don't understand what I am doing, this is not meant to criticize, that is why I am here afterall, for you--or someone--to finally understand how this works.  I am not looking through Pi for data that matches what's in Pi.  The numbers in Pi are irrelevant.  My formula uses Pi or any irrational number we want to lay a path like Hansel and Gretel through the forest to the Witches House.  The bread trail is merely dropped down when we encounter something important.  All other numbers are irrelevant. Imagine Hansel looks for trees with mold growing on them (0s) to drop his bread crumb.  And Gretel is looking for magic mushrooms (1s) to drop her mushrooms.  We begin our journey into the forest, with every tree in the forest one of the numbers in Pi after the decimal point.  As we travel along, pass every tree one by one, looking for mold or mushrooms.  When we find a moldy tree (the 0 we are looking for in our current byte to be encoded (say 00100001) we consider that first 0 encoded right there.  Now our second bit in our string is a 0 too.  So we progress to the next moldy tree.  We skip everything else along the way.  We hit our next moldy tree, and have now considered our second bit encoded.  Now we are looking for a 1 bit, so the thing we are hunting (currently 4s in Pi) are changed by +1 to 5s.  5s are now magic mushrooms.  Since bits 4,5,6,7 in the example above are all 5s, we keep hunting magic mushrooms for 4 hops (hopping on the 5s now) and on our last bit (another +1) we are hunting for 6s, clearings.  Oh, now we've completed our hunt, we have arrived at the witches house in the clearing by the end.  So our Crypto Key is now  [1.0.174.6.0]. 

If you sent that Crypto Key to me as I have just displayed it there, I could go into my Pi index I've built (it's very small, but helpful for under 5 bytes or so), and count backwards from there until I found the starting point and from there, I would know all the data going forward merely by reading the changes.  In the Crypto Key above, here is the breakdown of how a Crypto Key works backwards to decipher the Timeline.   (timeline means from decimal point forward thru Pi to Ending Index #.)   [1  (the first number) is the Mega Chunk number    0  (our second number) is what the starting Bit was (0 or 1) so we can be sure of where we initially began looking.  If it was a 0, we start with 4s but if it was a 1, we start looking for 5s right from the start.   174 (the third number) is our Ending Index Value of Pi.   6 (our fourth number) is the End Hunt Value, which tells us for sure that we were hunting for 6s when we stopped encoding.  and the final 0 (our fifth number in the crypto key) is our Flip Count. This tell us how many times we cycled from from our initial hunt value (4 or 5) back to that same number. 

Our Flip Count works from our first Hunt Value back to itself.  5 to 5 ....  so in 2 bytes of data, if there were eleven 1's, our ending Hunt Value would be 6 and our Flip Count would be 1.  It would look like this    [1.1.?.6.1]   Meaning First Chunk, starting bit = 1, (? = ? because I'm not looking it up now), Ending Hunt Value=6, and Flip Count = 1 (it flipped once when we passed 5 the 2nd time in our example 2 bytes of data containing eleven 1s).

C) The index isn't included in the file we save, that would be stupid, since it would do just as you say, include a lot of data we don't need to even include.  The index is Pi, a known number anyone who programs can program in moments to auto generate, in less than 51 bytes I'm told.  So our co/deco would include it built in to generate our index is RAM.  The part you're not getting (and again, I don't blame you, I WANT you to understand because that would be awesome!) is that we don't include any index in our output file, THAT'S WHY IT CAN BE 4K.   The fact that we used an index in Pi that was 16 million indexes long has nothing to do with our final file size, because we are just writing down the Index point itself. 

C part 2)  In addition, our Index is also more complex in that we are not going to run one program all the way through Pi to obtain the Crypto Key unless it's under a certain size (to be determined later, with research) and we will not allow this program to encode anything under 500KB in size also.  When running 8 GB through our program, we will use the same 2GB of RAM (with Pi written into memory there while the program is running to encode or decode, but dumped thereafter) and run that program in equal pieces.  Our 4k program will also tell the last piece (if it is not equal) exactly how many bytes and bits down to the bit it needs to add (which will later be removed) in order to keep all 4 pieces exactly equal.  That equal size is also written into the 4k file, so the program knows exactly how much data it's working it.  Thus, an 8 Gig file could look like this:

[OPENFILE]
[FileSplitSize=2GB]
[filesize=8000000000_&_Name="8GBofStuffForWhomever.zip"_&_LastChunkDataSize=956,524]
[MChunks=4_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[2,w, xxxxxx, y, zzzz]
[3,w, xxxxxx, y, zzzz]
[4,w, xxxxxx, y, zzzz]
[CLOSEFILE]

I would LOVE to take your bet, I really would, but I am not a programmer, nor do I have elite math.  I have no hope of programming this thing myself.  All I did was invent the process that needs to be taken and implemented. If anyone on this forum wants to program my idea for me, then I'll give you anything you want (within fairness, as long as I get some money when we strike it rich, and as long as you keep my name as a co-inventor.  I say co-inventor (not full inventor) because that programmer is going to need to be able to figure out how to read from my index point backward and solve my timeline to decode the bits out of Pi again.  If they can't do that, its all for naught, and I'm sure its going to take some hard math and genius of his own to accomplish that.  I may not be useful for that portion.   

Then we (those of us working on the software) will take your bet, and we will also take the Hutter award too, because when we encrypt that 100 MB wikipedia document down to 4k, they are going to owe us the full 50,000 Euros, too. 

Yes!

And Yes!!

And YEEESS!

AND YEEEESSSS!

God, I love you BurtW!  Thank God for you.  You get it.  You know I am not joking around here and you see it more and more clearly every day.

I NEED a good bitch slap to calm down after arguing with you all for so long!  Hahaha ....   

Now I got a programmer in my team, I just wonder what is the workflow of the compression(reduction) or encoding or whatever term is appropriate?

Can you give us a simple workflow like in this format:

Begin -> Analyze -> Encode -> etc. -> etc. -> End


Best Regards,
Balanghai

That is exactly your problem. You are right, you can just index everything in pi or any other endless continuing non periodic number. BUT like everyone here told you, it is simply not possible to do that in an efficient manner.
I will give you a little example:
Say you have a file the size of 10 MB: now you read out the actual 0's and 1's in it and search pi (or any other) for the representing string and store the starting and end index in your file. So far so good, now you have a file which is only the length of the starting and end index. That is really small, you are right. BUT you have to get the corresponding string in pi, which takes A LOT of time to find first.
AND everyone who wants to recreate the file has to compute pi UNTIL he is at the corresponding string again. And that is the problem here.

Let's say you have around 500mb of data!  If you round it a little you have a corresponding string of 4*10^9 bits length.  Now, to make that a little more efficient we convert it into the dezimal system, since a string of 0's and 1's that long would be way too long to search in a dezimal number.
To make it easier say 10^9 binary is 10^3 in dezimal. so now you have a string of numbers which has a length of 4000. Now, you need to find this string in an infinite number. Now you could use BBP to fasten your search over multiple processors and you would still search a VERY long time for a corresponding number.  

I just saw your last explanation. But that changes pretty much nothin on my statement above.That is not a new theory and it is proven to be not efficient in first year university information technologies iirc.

This is totally awesome work, Onkel Paul, I truly appreciate this kind of thinking (I hadn't thought of a chart like this to help visualize this), and that you also understand the rules thus far.  

The only critique I will make of this is that again, the theory is not meant for small data sizes, and using an example like this might not work to describe the theory's effectiveness because as the size of the file diminishes, uniqueness diminishes with it.  And a 1-byte example is also small than the 4k output size of the file in question, meaning it's inversely efficient to my design proposal.  

Here is why:

Imagine if I take 10 hops.  In just 10 hops (which is essentially the beginning of Pi) we are bound to hop over the same data with various 4 bit examples because there hasn't been enough room to yet establish uniqueness.  If that's the case, perhaps we need to capture the first 64bytes from the original data to overcome this problem.  The output could look like this:

[OPENFILE]
[FileSplitSize=1GB]
[filesize=8000000_&_Name="8MBofStuffForWhomever.zip"]
[BaseKey = 01101100110110011100000110000101101011111010..... to 64k]
[MChunks=1_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[CLOSEFILE]

The Basekey size might have to be at least 64 bytes I think in order to get around the counting problem observed in your arguments thus far ....  So that would mean the first 64 bytes would get added to the file, increasing it from 4 to 64k.  That's sad, but still doable.  The longer the data stream, the more unique the outcome.

Now back to WHY:

A byte is 8 bits long.  But traveling into Pi initially, we are overlapping due to there not having been enough time to uncover some 1s in the data, which are far more efficient than 0s.   Look at 1111 in your table quoted above.  It stops way shorter than the other 4-bit sequences you've shown there.  We must incorporate enough initial distance into the program to account for there being no possible overlap.  As we travel further into Pi (10 bytes in, 20 bytes in) a larger distribution of 1s in the software we are encoding would have begun to compress that timeline, making it more efficient, while the same sized-but-non-identical file comprised of more 0s would have gone further into Pi, meaning all of its branches would be on roads not possible to be taken by the other version of the file.  These small size examples can be counterintuitive to the concept.  But now thanks to you, I see that as we start from the decimal, there are too many possible overlaps, and thus we need to capture enough of the first part to account for those overlaps, however much that turns out to be.  Because at some point, it will be that the programs taken enough twists and turns to have brought it into a different trail entirely by the number of hops taken.

In other words, we need someone brainiac-like to calculate the following:

Given 8 bits to a byte, given 0s being less efficient (using my theory) in Pi than 1s, how far would we have to go in Pi before the number of hops taken would create a unique remaining road?

Here is an example for you:  

 1 byte               Index Ending in Pi At:  
00100010                      103              (the least effective 4-bit sequence doubled thru Pi)
11111111                       67               (the most efficient 4-bit sequence doubled thru Pi)

At some point along our path thru Pi, that while we may still hop over the same digits, the number of hops taken to get there will have differed, and that means we can now break our timeline with two different files of the same size.  Breaking the timeline means that route through Pi cannot produce our file again, meaning its more likely that there will be only one unique route back to our BaseKey.

You guys are becoming quite awesome now, even the hecklers are starting to turn around. Maybe one day I'll even convince PaulMurray there is something here worth looking at.  Until then, I must keep my nose down and incorporate your teachings and ideas.  THANK YOU GUYS!

Teleportation Concept Worth Big Money - I Did It!

I am a normal guy etc etc and I thought of a way for teleportation. I can't prove it works since I can't code, but the idea seems to work in theory!
This brilliant idea can change the world! It can be used for space travel, transportation and so much more.

1. Scan human
2. Convert atoms into code
3. Compress using B(asic)Miner's method, since there is too much human code
4. Send 1 line of code to the receiving end
5. Decompress and rebuild human

PLEASE GIVE ME A CHANCE, I JUST NEED SOMEONE TO BELIEVE IN ME

But the problem does not go away when more bits are "encoded", it only gets worse!
After just 4 bits, the encoding machinery is in the same state for some bit patterns, and it has therefore lost all information about which of these patterns it had encoded. The same is true for any 4-bit (or longer) subsequence in the input file. If you have two files that are identical up to that sequence, the encoder will be in the same state with both of them before encoding that sequence, and there will be groups of possible bit values that will all leave it in the same ending state after the sequence.
Going further into Pi before starting the process does not help at all, the distribution of decimal digits is uniform.
Adding the first 64 (or 64k, what's a factor of 1024 between friends?) bytes to the encoded file does not help either - it just shifts the point where the non-uniqueness problem appears 64 or 64k bytes into the file.

Please realize that wishful thinking does not heal a scheme that's fundamentally broken.

Onkel Paul

(I know I'm too old to be trolled, but at least it'll increase my activity, which is at least something...)

You made my day. Thank you.

I encourage newcomers to the party, I'm all in.  But you would need to go back and re-read a large portion of this starting from about page 6 forward until now and then you will see that what you're talking about is compression, and thanks to BurtW, what I now know this to be called is creating a Encoder/Decoder that allows you to pull data out of Pi with a Meta Data keyfile of between 4k and 64k.  


BALANGHAI

Essentially, this is the entire process:

1) Open file to processed.  Analyze the data.  Then we open our file and begin to record the following:
   A) Original Filename.  Size of the Original File.  Size of Pi Index Used (how big each chunk is to be split into).  Size of the last Mega Chunk in bits (if applicable).
   B) Basekey (the first 64 bytes of the original file, giving the program enough room to establish a unique path given a number of hops.
  
2) Begin reading the data one character at a time (converting hex to Ascii Binary) all in memory using the loaded Pi Index to the Pi Index Size shown in example A above.  Convert everything (all file contents) to Ascii Binary (so every incoming piece of data is exactly 8 bits).  Begin moving forward through Pi by hopping on a single solitary digit called a "Hunt Value" meaning the number we are to be hopping on in Pi.  Starting from the decimal point, begin encoding by hopping.  Hop to the 1st 4 in Pi if our first bit is a 0.  If it's a 1, hop to the first 5 in Pi.  Hop along Pi, encoding 0s and 1s using these rules.  0 = no change in Hunt Value.  If you start with a 4, you keep searching for the next 4.  1 = +1 to Hunt Value in Pi.  I'm sure this can be done in realtime with data, you are just moving along, not having to do any hard math at all.  Computers were made for this kind of math, it's the most basic, so our encoding would be lightning fast.

3) When we reach our size limit for our first chunk and there is more data to be read in, we open our file and write our first MetaData Key.
   C) [1.x.yyyy.z]   (for those who have been following, we no longer need the first bit, since we have added something called the BaseKey, a 64 bit record of the initial string of the file.)
Keep encoding until all the data is complete.  Record the size of the last chunk into the file record in bits, and add 1s from that point out so the last chunk is equal to the other chunks exactly.  During decompression, the program will compare the data split size to the last chunk's size, and remove the 1s automatically when it comes time to write out the last file.
Close the file out and dump the Pi Index from memory, clearing up the computer.


DECODING:
Start from the Ending Index Location.  Begin hunting through Pi using all combinations of 1s and 0s in some systematic or mathematical way, looking for efficiency and intelligence in solving how to do it the most quick possible way, to find the one path through Pi that ends at our Ending Point exactly using the same exact number of steps, and calculating intelligently the number of 1s encoded in the formula using the Flip Count information (so we don't waste time searching for paths that have more 1s than are even in the timeline we've created.  Since any fork in the road will shrink and compress a given file, the overall chance of that file having the same 64-bit Base Key, number of steps to reach the end goal, and same number of 1s, would be significantly reduced.  Add to that, since we are not encoding the file all the way into Pi for one single MetaData Key, but splitting the files up into more and more MetaData Keys, the chances of the file's uniqueness will keep going up with more divisions, while the number of Metadata keys added to our final file output would create a larger and larger file.

It my goal to be able to make this work with 500k files up to 5 TB files using this system if it can be made to work.

Okay, guys, you got me.  Good one.  Although I did say in my first post (jokingly) that this could work for that one day, when the technology reached a high enough level.  And how do you think we are going to ever get there, if we don't at least dream and attempt to do that?  Maybe my idea seems ludicrous to you, but I'm sincere, I'm trying to do something cool, and its the best I can do.  I'm sorry I'm not Bill Gates.  Oh, sorry, I forgot, he was only a businessman who ripped off everyone else and gave himself credit for it.  But I admit he's a lot better than me.  In fact, I have to sit here and use the computer his company made and the OS his company made to write all of this, so that pretty much settles it, right?

I don't deserve to even try, is that?  I should live in China teaching children my whole life, making $1400 a month, and never dream of anything, just stare into space my whole life.  At least a few of you seem to care about what I'm trying to do.  I appreciate that.  It's hard having one's best efforts shot down.  I never said I was smarter or as smart as anyone here.  I just want to try.  TRY, damn it all.  It's so fun, and it might make a difference one day.  Which means my life might account for something.  It's not my fault I was poor all my life, and taught to hate school until I was almost 35 years old, by then it was too late. 

Life is harsh.

You are right, life is harsh. Nobody will believe people like you and me. They think we are fools, or clowns.
We must follow the examples of great pioneers like the Wright brothers or Thomas Edison. We will change the future, and we will be remembered.

Fall seven times, stand up eight.  ~Japanese Proverb

Like I said before. This changes nothing. there are several factors that condemn this to fail. Even if you could solve the uniqueness problem, which you can't, there are still several factors. your "chunks" are good if you want to shorten the jumptime; you need a chunk everytime you reach a certain value, say 33 for the heck of it. so pretty much every 33th bit you need to write the corresponding chunk in your file. the higher the value the longer you have to search through pi.
So, now here is the problem: you want to continously iterate through pi, which is fine by itself - BUT you have to store all that in a memory. you could use certain formulas which enable you to jump to a specific place in pi to shorten the amount of time you need and memory used, but we are talking about a supercomputer here to effectively iterate trough pi, get the corresponding values, write them in a file, remember where you where/use a formula to get there again, do that over and over again.

Don't misunderstand me - You had a really nice idea. But you are not the first to try that. The main factor of a Decoder/Encoder is the speed. You don't have any speed.

To get you to understand this is pretty hard, you don't seem to read the numbers correctly.
I told you that 500mb is a string with 4*10^9 numbers. 5TB?!?
That is a string with 4*10^13 numbers. You need to iterate through pi at least forthy TRILLION times, and that is IF all the hunting values are right behind each other!
This won't change even if you always start from the top of pi!

Yeah, exactly.  Do you know Gustav Whitehead?   No?  Nobody does, but some people think he invented portions of the airplane before the Wright Brothers, who were credited with having invented the airplane, but actually they only improved upon a number of other inventions done by others, by putting it all together.  Nobody remembers Whitehead, but he and others like him paved the way.  The problem is, they gave up ... and the Wright Brothers didn't.  And the rest ... is history.

I hate to destroy your dreams, but your idea is fundamentally flawed. As has been pointed out repeatedly in this topic.

In your first post you wrote:
This is simply not possible. Not a matter of requiring faster computers, more memory, or better technology, but a matter of theoretically, mathematically provable, fundamentally, logically impossible.

You keep beating around the bush and getting lost in irrelevant details, thus deluding others (and possibly yourself too) from the very clear, simple fact that you are missing a crucial point.

Misuse of terminology causes some confusion here, what you call 'index point' is what others call 'index'.

Using your terminology, let me rephrase the essential flaw in your approach: for pretty much ANY file (aside from 0.00000000001% very lucky cases), the index point (or 'crypto key' or however you call it) required to re-create the original file, will require MORE data than the original file itself.

The fact that you can re-create Pi (or other irrational numbers or infinite data sets that contain any possible file in existence) in just a few bytes, does not change this fact in any way.

The stuff that Whitehead invented was not theoretically fundamentally logically impossible. It just couldn't be done in practice at his time, because human technology was not advanced enough yet.

Your idea on the other hand, can be mathematically proven to be wrong. No matter what kind of smart thinking, advanced technology, quantum computers, zero-point energy, or other hypothetical fancy stuff we'll have at hand in the future. Math doesn't lie.

Well, if you mean the time needed to load 2GB of the Pi Index into memory, I assume that would take 1 minute or so to, upon starting the software, generate the Pi Index (of that size) needed.  Who cares if it's in memory?  I don't.  Besides, who knows if I will end up using 2GB, it could end up being 500MB of Pi, with smaller chunks.  First we need to just get a basic 2 meg version running, and try to encode files between 500K and 1 MB and just see what happens.  

But if I get this working, imagine the time saved over the internet to send your friends or family the 20GB of videos taken on your vacation.  You would be sending a small file 4-64k large, in their email in moments.  Then they'd decompress out the videos overnight, while sleeping perhaps.  Wake up, the videos are there.  The internet did not need to be congested with all of those 0s and 1s.  And if a lot of people were using it, the internet would work more and more smoothly all the time.  Think of the difference!

Another thing is, you still can't convince me that just because it's possible to have 2^N files TO encode, that there are that many unique files.  For all we know, our research into this could reveal a fundamental characteristic of Nature to only allow organized patterns to assemble in random data at a given ratio, like the Golden Means (8/7 is it)?  Just trying to solve this problem itself could lead to a breakthrough.  What if there are only (2^N)/10 file in existence of each size, and they already happen to be written in Nature?  It would mean all of our ideas already exist in the Global Intelligence of Nature and that our brains are receiving the information via DNA alterations that come from eating plants.  Because science just recently confirmed that by eating some plants, our DNA is altered, and they hypothesize that new ideas come from this phenomenon.  If Nature is the one providing original ideas, it stands to reason the answer is already in Nature for every created thing.

I don't wish to go too deepy into philosophy here, though.  Just saying, you don't know for sure that even though there are 2^N files possible, that all of those combinations will ever be used to organize intelligent data into a file that ends up on someone's computer that could be put through my system.  In that case, there might be all the uniqueness I'll ever need.  Won't know until we try it and watch it either (Get busted) or work as conceived.  

B(asic)Miner, please answer this:

You claim for any input file, you can create a 4K 'crypto key', from which you can re-create the original file.

Suppose we take two different files that are each 5K large. Let's call them A and B. Through some smart process (whose technical details are irrelevant for now) we calculate their corresponding crypto keys which are 4K each, i.e. two pieces of 4K data. Let's call these P and Q. Now, if it's possible (by means of some other smart process that may or may not involve generating Pi or whatever) to reconstruct A from P, and B from Q, do you agree that if A and B are different, then P and Q must be different as well?

(If not, i.e. if P = Q, then reconstructing the original file from P (or Q, which is the same) will either result in A, thus we can never re-create B, or it results in B, thus we can never re-create A)

Okay, then I ask you to explain to me how that works.  Because it sounds crazy to me, it sounds backwards, it sounds wrong.  So I must not understand what you understand.  So teach me, if you will, so I can try to understand where you're coming from with this.  

For example, let's say I want to send you a quote from Alice in Wonderland, but both you and I have the book in our library.  So instead of me trying to send you the PDF thru the internet, I think, hey, why don't I just send a reference?  (What you're calling the Index)

So I write:  "Alice in Wonderland, page 15, lines 1 through 3."

Then I think why send all of that when I can just send this:   "AlceInWndrLnd... p15, Lns 1-3"   so now my Index (cryptokey, whatever) is just that (and it's only 28 characters long).  

You open the book and find the 1st paragraph.  It's 3 lines long.    It has 52 words, or approximately 280 characters.  That means that my reference (index, cryptokey --whatever) is only 10% of the size the data you are now reading is.  I've just saved 90% of the data over the internet than I would had I just copied the text off the internet directly.  Sure, it took you a minute longer to go find the book and bring it off the shelf, open to the page, etc... but my fundamental concept is just the same as this.

Now explain how my 28 character is somehow supposed to also include the book + the 28 characters over the internet when I clearly never sent the book!  Its for this reason, I don't think you understand how my theory works fully yet.  Not a flame, though, trust me.  I want to both understand you, and you to understand me before all hope is ruled out ...

*sigh*... that's not what I'm saying.

Yes, for these particular 3 lines, this "Alice in Wonderland based encoding (or compression) scheme" would work.

However, for 99.99999999999999999999999999999% of all lines in existence, it wouldn't. Or it would require more index points or more references throughout the book, ending up taking more space than the actual line itself.

I give up. You are just too dumb to understand what people are trying to tell you. What you wrote up there is utter bullshit, in every way. there are pretty much uncountable unique files out there. Which still has nothing to do with it - at all! You don't even seem to understand that each chunk you want to save has a specific value itself, and you need a lot of chunks. Your idea of a pi string is nice, you don't have to compute it if you have it already stored somewhere. you still have to iterate through it. A LOT!


Since pi is an INFINITE number all of the data in the world is stored somewhere in there. Along with the cure for every illness, every word ever said, every formula ever created. so yeah, there is pretty much everything written in nature. That is nothing new.

My whole post was never about the number of files but about the fucking length of a single 5TB file and the amount of storage you need for the chunks, the memory and the freaking amount of operations you would need for a single file to be processed that way.

But hey! With a quantum computer your idea would be brilliant! But then again,you could just index the complete file in pi by then, since it would be there instantly.

This isn't even necessarily true.

The number 0.121221222122221222221etc (that is, its decimals consist of infinitely increasing sequences of 2s intersected by single 1s) is also "infinite", and irrational, yet it doesn't contain "3" anywhere.

<.< I did not state that this is the case with every irrational number now, did I?

Not trying to get off topic here, but you said "Since pi is an INFINITE number all of the data in the world is stored somewhere in there." and this argument is simply incorrect. Being "infinite" (what does that mean by the way.. not having a finite decimal representation? would that make 1/3 "infinite" as well?) by no means implies that all of the data in the world is stored somewhere in there. As my counter-example demonstrates.

As for Pi, is it actually not known yet if all data in the world is stored somewhere in there, or not.

I hope you are not setting me up for some math trick just to shame me.  I will try my best to answer this, but I feel I am walking into a trap.  I will give you the benefit of the doubt.  First, although I've taken extensive IQ tests online, and all my results hover at around 158 (some 145, one said 163), that aside, I barely graduated high school due to my body's reaction to stress.  I was a slow learner, a deep thinker yes, but slow because I had disabilities due to my strong emotional nature, whereby the slightest provocation would create intense emotions which would shut down my brain and cause me to be unable to think.  Being in school was always provocational, there were always bullies, and I was weak, a toe-headed small-bodied wimp, begging to be picked on.  My own home life was filled with step-brothers who saw me the same way.  I had no chance to learn to enjoy school or my own home life either.  Whenever I try to think today (as a legacy from those terrible early years), I get nervous, unless I'm alone, which causes me to slow down and become almost moronic in public, saying the most stupid things out loud.  So please please don't sandbag me with some math trick.  I'll try my best, but if you're intent was to destroy me in some way, please reconsider going through with it.  In the off chance that you truly want me to understand something important here, I will try to answer you.

First, my theory won't support files that small, its pointless to even bother with files that small, so I'm changing your input to 5 MBs, okay?

Each 5MB file A & B gets reduced to 4K, becoming P & Q.  As far as trying to now use another method to recapture the data, I must say sorry, there would be no other viable method, since it was sent through Pi, it must come out through Pi.  We could encode the same A & B files thru 2/3 or another irrational number but we would get different Metadata (Index).  

I agree that if A & B are different, P & Q must be different as well, yes.  

If P=Q, then what you have are two identical files, which can't be P & Q anymore, they would only be P.  Q would just be another copy of P.  That would mean A and B were both identical to start with, and in the beginning of the math problem, you said they were different, meaning P cannot equal Q, therefore this problem becomes irrelevant.

Please don't get angry, this is how I see it.  Maybe I don't understand.  I apologize.
  

This argument is hard to beat :-]]].
Of course there ARE NOT (exist somewhere on disk, in memory, written as book) 256^1024 of 1 kB files. But theoretically there could be.
What you are basically saying is that you algorithm will work for all "reasonable meaningful files". There is indeed very low number of them. But this is out of the scope of mathemeatics, because you can not define what exactly is "a file that means/will mean somethinh reasonable.

But all 2^N files can be created by simple computer program. (For some very high N bellow some limit dictated by current technology.). There is no known fundamental characteristic of Nature which prohibits some combination of bytes in file!
Or do you believe that some of 256^1048576  1MB files are physically impossible to create (save, copy, print)???

No worries. Just trying to either prove you wrong, or be proven wrong myself, through a logical argument. Not trying to trick anyone, merely hoping it will bring clarity on the subject.

Yep, that's fine.

OK, we agree so far. Now, in order for this approach to work for 5MB files in general, any 5MB file we try to compress (or encode) must be reduced to a unique 4KB crypto key, right? (otherwise, we'd have a conflicting situation where two different 5MB files A and B would reduce to the same 4KB crypto key, which we just agreed was not the case)

By the way, when I said "by means of some other smart process" I just meant the decompressing or reconstruction part does something else than the compression or encoding part, which was kinda obvious (in fact they complement eachother), but never mind, it's not important for now.

I agree with that partially.  It doesn't mean all data is stored in Pi in easy to read linear fashion.  But all of life is made up of patterns, and for sure in every software program there are redundant expressions used over and over again by coders who want to maximize their time by copying old chunks and familiar lines of code that once made them famous in their fields.  Fundamentally, all of programming code, when rendered down to the most basic computing language is just binary, just 0s and 1s.  I am not saying people who think they can use strings and then hunt through Pi to locate those strings and turn it into Metadata are on the right trail.  I think they are not.  It's too slow, for starters.

My idea produces a thumbprint based on a file that is already created.  Trying to enter just any random Metadata (what I call Crypto Key) set into my software would only produce noise unless you just happened to stumble across a set that was already in there.  I think that's the real reason, if only on a subconscious level, people are against my idea, is that this notion is terrifying.  Imagine putting in a random Crypto Key and spitting out a video of 2016 showing Russia invading the U.S. just like in Red Dawn the movie, only for real?  That would violate time and space.  It seems a bit terrifying, but yes, I've thought about that possibility, and once when I thought about giving up on my theory, I almost decided to just write a short story about this instead.  But the desire to actually create something that does truly work is way too strong yet.

What is important is that whatever data you do put into it, produces a Metadata set that only works with my theory.  But if you change the base Index structure (not Pi but another number) you get different Meta Data.  Perhaps the thing to look at here is that Pi cannot contain everything, perhaps Pi only contains 1/42nd of everything.  Perhaps my theory needs to have multiple irrational numbers.

What if there are 42 irrational numbers and each of them contains part of the whole of the collected data and what if we need to process certain files through different irrational number schemes in order to capture the totality of the uniqueness needed to break the counting argument?  Perhaps one data set is too small to encapsulate everything, but all irrational numbers combined, it would?  The problem then would be in trying to figure out what files need to go to what irrational numbers for their uniqueness to be preserved?

That sounds like math I think no one could achieve.  But then again ... I won't say it's impossible because I don't believe in that myself.  Everything is possible somehow, if we only knew.

Sorry, what's there not to agree with? Either Pi's decimal representation contains any possible sequence of digits, or it doesn't (i.e. there exists at least one specific sequence of digits that does not occur anywhere in Pi's decimals). Which of these two statements is true, is not known yet. It has neither been proven, nor falsified.

I think that I can pedict your next argument and the next argument of B(asic)Miner :-]
Kazimr: Imagine long column of all 5MB files on the left and long column of all 4KB crypto keys on the right. Then connect with line which file on the left gets compressed to which crypto key on the right. Soon you will run out of crypto keys and some lines will point to the same crypto key...

B(asic)Miner: Most of the files on the left will never need to be compressed as they will never be produced in this Nature. (Because Nature allows only some reasonable/natural patterns.)

Do you believe that it is "somehow" possible to find the largest prime number. Such as no larger prime number will ever be found?

Yes, I agree.  But only partially.  If we included a function of the software that did the following, it might still work:

"I'm sorry to report, but your key has resulted in 12 collisions.... wait.... accessing the Basekey information ....  unique file found from key."

Now, although there were 12 collisions (where other files started and ended at the exact same place, had the same number of 1's in their files structure, and otherwise fit all the criteria, 11 of them had at least 1 bit different in the initial 64K Bit Key included in the 4K file, so the program was successfully able to weed out the other 11.   But let's say instead of that, we get this message:

"From unique key sampling, there are still 2 collisions, what would you like to do?"    You access:  "save both to disk."  Now you have two files on your desktop.  You try the first one in your videoplayer and it doesn't work.  You try the 2nd, and your movie is playing right before your eyes.  You take about an hour to try and see what the 2nd file is ... you try it as PDF, Doc file, Zip file, audio, everything, finally you discover it's a 3DS Max scene file from some guy in Florida who does 3D work for some studio there.  Amazing, it's exactly the same size as your file down to the last bit, but it's an actual working file that someone else made.  Now you have to wonder:  who made the file?  That man, or Nature?  Nature came first.  And it's numbers don't change.  Maybe the man discovered the art he is making through accessing Nature?  Creepy.

But perhaps this is how we solve the overlapping argument once and for all:  we let the software detect collisions and give the user options about how to work with them .... 

WAIT:  OMG!  This is the answer!    We use the collision detection DURING ENCODING AS WELL.  This would dramatically increase encoding time, because you'd have to decode the file to see if there were collisions before allowing it be used as your save point.  You could try a list of irrational numbers until you found the one that gave you the unique ending spot!  It's so awesome!   (But yes, slow!  Very very slow!) 

Or just allow for the Collision Detection after the fact and work with some tricks to try and pair down the collisions found.

Infinite means per definition not ending. 1/3 is not infinite since 0,33333333333 (periodic) is finite written as 1/3. Per definition 0,99999 (period) is defined as 1. Periodic numbers are seen as finite numbers in general. Pi is seen as infinite since it is an irrational constant. Which means there is an infinite amount of numbers contained within it -> ergo my statement that every data is included there. But you are right, I can't prove that. But you can't prove the opposite ;)

And for the OP:
Please read into the stuff you want to do a little further:
http://mattmahoney.net/dc/dce.html#Section_11
http://www.maximumcompression.com/index.html

Kazimir:  YOU SIR, ARE GOOD.  I had a great laugh over that one.  You got us both dead to rights.  And I knew it was a math trick, but at least you took the humorus approach to it rather than the "AHA! GOTCHA SUCKA" approach.  Too funny!   That's exactly what I would have said, too.  Or tried so say,  You said it better.  That is also funny.

That is a pretty good idea there.

42?  Are you a Hitchhikers Guide fan?

hmmm...

The problem with your "Partial Agree" is that there will be not 12 collisions, there will be infinite number of collisions.

Is this a real coincidence ?

Compression with Pi (http://encode.ru/threads/1791-Compression-with-Pi)


A forum topic created in encode.ru yesterday.

Sure, only for very large files the message may be more like:

"I'm sorry to report, but your key has resulted in 12,234,123 collisions.... wait.... accessing the Basekey information ....  now there are only 10,234,123 possible files."

It's not a coincidence ... but it makes me extremely nervous.  Because I already posted my same idea over there and now someone else is trying to claim he had the idea, when I posted on the very same forum the same day I posted here.

Check it out here:    http://encode.ru/threads/1789-100-Lossless-Compression-Theory-Please-Join-Me?highlight=SmallestFilePoss

I B(asic)Miner am called SmallestFilePoss on that forum.  You can check my Skype and Yahoo information on that site for cross reference that I'm telling the truth.

Sho is this Mexxi joker?  Is he someone from this site who is just messing around with me, or is this guy for real?

Mexxi's responses on that forum seem very reasonable.

For Pi, no, I can't. But I have given an example of another irrational constant, which also has an infinite amount of numbers contained within it, yet some numbers are not in there.

Hence the argument "x is infinite, thus every date is included there" is not true in general.

In fact, if a number if infinite, anything is possible:

Number	Infinite?	Every data included?
1/3	no	no
0.121221222122221etc	yes	no
Chaitin's constant (http://en.wikipedia.org/wiki/Chaitin's_constant)	yes	yes
√2	yes	unknown!
Pi	yes	unknown!

Just curious:  what about e?

You guys, check THIS out!  I can't believe this, and it's on Wikipedia, too!

https://en.wikipedia.org/wiki/Jan_Sloot


Read this!  It sounds like he thought of the same thing I'm thinking of ....  holy cow!

Well, instead of a "math trick" I'd rather call it "sheer logic".

Anyway. Do we now agree that your idea is not possible? If not, then do we at least agree that this:


must be true in order for your idea to be possible?

I think its pure coincidence

Not known (yet), just like pi.

Jan Sloot was proven to be a fraud long ago, yet there are still "believers" who prefer to ignore common sense and defy logic.

It goes against my own interest to publish this here, but I will do so because of the hard work of the people who have followed me so far.  The man sounds a lot like me.  If he hadn't of died in 1999, I would have I was his reincarnated self trying to do this all over again.  It is strange he died on 9/11 1999, the year many think the government went online with some kind of technology which could see into the future which told them that 9/11 was going to happen if they wanted to take control of the world (which they do, so it did) ...  I'm not saying I believe that, but what if his idea was possible, and it's all been covered up?  That would mean by trying to recreate it (although I had no knowledge of the man before today) I would be asking for trouble.  *I* might have a heart attack if I try to create this ... if you get me.  Maybe some corporation (a data storage corporation) doesn't want this secret getting out which would bankrupt them?   Jeesh, I am becoming as paradoid as this man.
--------------

Translated from UT News:

In 1999, Jan Sloot, a common television repair man, discovered a revolutionary coding system that would make hard disks, CD-ROMs, and other data stores superfluous. All movies ever made would fit on one CD-ROM. Sloot attempted to sell his invention to Philips, but he found that Philips' engineers were not interested.

Roel Pieper, at that time a member of the board of Philips, thought the invention was valuable, and decided to join the inventor and his companions. Pieper and Sloot looked for investors all around the world. Pieper himself invested millions in the project. All investors of The Fifth Force, Sloot's company, hoped to become billionaires.

But the paranoid inventor died on September 11, 1999, one day before the invention would be specified in detail in a contract. All his notes, his prototype, and his source code were lost forever.

Translated from GIDS:

The Sloot Digital Coding System (SDCS) would shake the world: a new alphabet for digital storage that didn't use binary code, but a much more efficient method. The principle behind SDCS seems simple. As a text consists of a limited number of characters, a movie consists of a limited amount of colours and sounds. All those basic data were stored in five algorithms in five memory stores. For movies, each algorithm would have a maximum length of 74 Mb. That's 370 Mb in total: the invention's engine. To start the engine, only a proper key was needed. For every page of a book, for every image in a movie, Sloot calculated a unique code. The concatenation of these codes would again result in a unique code. The final code, the key, would be one kilobyte in length, regardless of the length of the movie or the size of the book.

Roel Pieper, professor in Computer Science, commented on people questioning the possibility of such a high compression (translated from UT News):

"It is not about compression. Everyone is mistaken about that. The principle can be compared with a concept as Adobe-postscript, where sender and receiver know what kind of data recipes can be transferred, without the data itself actually being sent."

So we're back to the alien and his stick. Basically, Roel Pieper explains Sloot's 'invention' as the movies already residing on the computers of both parties, but in a deconstructed form, and a key being transferred that specifies exactly how reconstruction must take place.

In principle, this is possible. But the crux is, how long the key needs to be, and how big the data store of deconstructed forms must be. Sloot claims that a key of one kilobyte, and a data store of less than 400 Mb, would suffice. This sounds unbelievable, and it is, in fact, mathematically impossible. There are many different arguments. I prefer the simplest one:

A key of a limited length (whether it is a kilobyte or a terabyte) can only store a limited number of codes, and therefore can only distinguish a finite number of movies. However, the actual number of possible movies is infinite. For, suppose it were finite; in that case there would be a movie that is the longest. By just adding one extra image to the movie, I would have created a longer movie, which I didn't have before. Ergo, the number of possible movies is infinite. Ergo, any key of limited length cannot distinguish every possible movie.

The SDCS is only possible if keys are allowed to become infinite, or the data store is allowed to become infinite (if the data store already contains all movies ever made, a key consisting of a number can be used to select the movie you want to see -- however, in that case it is impossible to have keys for movies that have not been made yet at the time the data store was constructed). This would, of course, make the idea useless.

The whole problem of Pieper's way of thinking (and of every other person who believes that Sloot actually could compress movies with a factor of two million) is that he believes that the key does not need to contain every detail of a movie. Unfortunately, it does.

In the SDCS, it is claimed that no movies are stored, only basic building blocks of movies, such as colours and sounds. So, when a number is presented to the SDCS, it uses the number to fetch colours and sounds, and constructs a movie out of them. Any movie. No two different movies can have the same number, otherwise they would be the same movie. Every possible movie gets its own unique number. Therefore, I should be able to generate any possible movie by loading some unique number in the SDCS.

Think of it: by placing the right number in the SDCS, I can not only get Orson Welles' Citizen Kane. I can get Citizen Kane in colour! Or Citizen Kane backwards! Or Citizen Kane where the credits misspell the name of Everett Sloane, who plays mr. Bernstein, as Everett Sloot! Or Citizen Kane in Swahili! Or Citizen Kane where 'Rosebud' is a teddy bear! Or Citizen Kane with a cameo of myself! Or the cartoon version of Citizen Kane! Or Citizen Kane where Charles Foster Kane wins the election! Or the gay version of Citizen Kane! Or Citizen Kane where Charles Foster Kane is replaced by Jar Jar Binks!

How many movies are possible that are variations on Citizen Kane? More than fit in a number of one kilobyte, I can tell you. More than there are atoms in the universe. An infinite number, actually.

In the SDCS, the details of every possible movie have to be somewhere. They are either in the number, or in the data store. Since the data store is the same for each movie, the details can't be in the data store. Therefore, everything that differs between movies, must be represented in some way in the numbers. Which, for infinite variations of movies, is impossible with finite numbers.

Would it be possible to store, say, all possible movies up to three hours in length, with a limited resolution and a limited sound quality, in a series of finite numbers? Yes, of course that is possible, since in the digital world, there is no difference between data and numbers, thus data of a limited length is equal to a finite number.

Even though Sloot claimed that one kilobyte was enough to store any movie regardless of its length, the question is warranted whether a system as defined by the SDCS is possible for movies of a limited (but realistic) length. The answer is no. One kilobyte is 1024 bytes, and a byte can take 256 different values. With a code of one kilobyte, to represent any movie up to 90 minutes in length, on average a byte must be able to distinguish between all possible sequences of about 5 seconds. Can a sequence of 5 seconds of any movie at all be identified with a number between 0 and 255? Of course not.

The concept of the SDCS, as described, is equivalent to a universal Turing machine. A universal Turing machine (which is actually quite easy to simulate in a computer), can emulate any possible computer program (including a program that shows Citizen Kane which ends with a pie fight) just by feeding it a number. The problem is that even for trivial programs the numbers quickly get enormous, so that it is impossible to filter the rare useful programs from the universe of chaos consisting of all numbers. Therefore, it isn't viable business to sell people a universal Turing machine as a solution to all their business problems.

Evidently, Jan Sloot was a crank. He showed all the signs attributed to cranks. He was paranoid. He felt himself an expert in a field he had no higher education in. He worked alone. He spent decades on one problem any mathematician could have told him was impossible to solve in the first place. He got angry at people who questioned him. He felt misunderstood. He had delusions of grandeur.

I think Jan Sloot really believed he had found a way to compress movies to one kilobyte. Probably he had imagined something as the alien's stick, something that would be theoretically possible in an alternate world, with infinitely divisible particles, but impossible in our reality. He probably lacked the mathematical insight to see that his 'invention' was a farce. In his prototype, he faked his invention, which is why he refused to let anyone near it, and answered only in mystical vagueness to questions. He probably imagined that, with enough money, he could get his invention to work for real. He probably did not feel himself a charlatan or a crook, but an honest businessman, who needed a starting capital to create the world's greatest invention, make some people incredibly rich, and win a Nobel prize.

What truly amazes me is that a man as Roel Pieper, who is a professor of Computer Science no less, could fall for his story, to the point where he invested a huge amount of capital. If his role in this story is really as reported in the media, his credibility as a computer scientist has been seriously tarnished. In my opinion, the University of Twente, with which Pieper is associated, should at least perform an internal investigation, to assess whether Pieper's position can be maintained.

Incidentally, if anyone is interested, I have devised an invention where I can store any movie ever created on a small stick. I need some money to develop this concept further. If you have a couple of million to spare, give them to me. I'll be glad to build a prototype. We're gonna be rich and famous!

Article Written September 28, 2004

Sweet Jesus, not that sh*t again :'(

I knew it.. In a thread like this, the name "Jan Sloot" was bound to show up sooner or later :-\

As someone with vast experience in mpeg video this quote made my day:


I never heard of "Jan Sloot" before today.  Thanks!

Because of this article, I can sort of understand a lot better than before, why you all have problems with my theory, or I guess why *I* should have a problem with my own theory.  Is there something about this discovery that is calling people like me out of the wood works to try and solve it?  Does the universe want this to be created at this time to solve a fundamental need in culture?  Our world is rapidly running out of room to store the vast data we are capturing every moment.  I want to believe this man had a true idea, but that someone didn't want him to bring that one forward at that time.  Does that make me a conspiracist?  The missing card that contained his program, his death, the date of his death, all seem suspect.  I guess you guys have all heard of this guy before, so I'm sorry to bring it up again, but wow.  I feel sort of deflated again after that.

I am truly trying to achieve what I said I am trying to.  I am not a crackpot or a prankster.  To those of you who have already done some work, such as programming small files to help calculate number is Pi, would you PLEASE UPLOAD those to me somehow, so I can use them myself?  The work you have done shouldn't be lost if I decide to keep plugging along on this.

The different between me and Jan Sloot is that I've shared the full theory with you, so now it's become open source, I guess. So maybe there won't be anyone coming to give me a heart attack, and maybe if I die anyway, then someone can still see how I was thinking and try to go it on there own.  But I doubt I'll get rich now, or even be remembered.  I've lost the chance to have done something astounding because now anyone who reads this forum entry will also know how similar I am to Jan Sloot and just go back to the "he's a wacko" argument.

Wow, that post reminds me of President Reagans favorite joke:

Worried that their son was too optimistic, the parents of a little boy took him to a psychiatrist. Trying to dampen the boy’s spirits, the psychiatrist showed him into a room piled high with nothing but horse manure. Yet instead of displaying distaste, the little boy clambered to the top of the pile, dropped to all fours, and began digging.

“What do you think you’re doing?” the psychiatrist asked.

“With all this manure,” the little boy replied, beaming, “there must be a pony in here somewhere.”

Let me just add this story before I call it a night:

When I was in the Army, I had a programming friend who was good with Amigas.  He created a videogame that we never had a chance to publish because someone stole his computer before it was completed, just like days before the completion.  We worked on it for months.  When creating the animation system for the game and the tracking system for the tiles and how they would move and animate, he got the clever idea of using text blocks to represent different aspects of the state of the tile.

It looked like this for one tile:   FDCBG  ....  five letters.  The first letter was the x, then y, then initial state, then animation frame on currently, and then several other factors.

To get the tiles to animate, he would append every tile on the board into one long superstring like this:

FDCCGFDCBGFDCBGFDCBGFDDBGFDCBGFDCBGFDCBGFDCAGFDCBGFDCAAFDCBGFDCBGFDCBGFDCCBG  and then use CHR$ RIGHT$ LEFT$  MID$ functions to grab out what was needed for processing.

Now that one string told the game where every file was, what it was doing, its animation cycle, everything needed to display a huge number of tiles on screen while simultaneously animating them at different rates, different start points, end points, everything.  This one code represented everything happening on the screen at one time.  It would grow and shrink as certain tiles got deleted, moved, replaced, etc...   I felt this was genius.

One small code could tell the game everything it needed about each individual tile.   Talk about data compression, talk about creating a system that could replace what would take other people hundreds of lines of code for each tile and be less efficient ...  it amazed me.  I think that's ultimately led me to find such a thing as this in the first place.  That was back in 1992, so I've been thinking about it a long long long time.

I just want one of two things:   1) for my quest to be over by being proven it can't work (in a public way, not in some secret board room) ... or 2) finding it can work and getting the reward for not giving up on it.

Thanks guys.

Your idea works. Just not like you want it to. It does not work on large files (5TB) and it will not work on small files (under a few mb). Other than that it could be a reasonalbe approach to data encoding. But again, look at the links I provided you, you will find a lot of stuff there, including efficiency marks for several compressng programs up to 98% compression for certain types of files.

B(asic)Miner, have you considered the possibility that maybe Jan Snoot (or whatever) really WAS just a crackpot? An ambitious crackpot, but a crackpot nonetheless?

https://i.imgur.com/pEAGXK5.png

Highly Compressed data (39 KB)

https://i.imgur.com/KUq33IO.png

Uncompressed data (39 KB)

1 pixel represents one bit
black pixel for '1'
white pixel for '0'

Entropy (http://en.wikipedia.org/wiki/Entropy_(information_theory)) can be seen in the first image.

A single 5 byte file: "Hello", results is many many more than two million collisions.
There will only ever be more collisions the longer the file is.
A 100MB movie file would result in more collisions than could be saved to a hard drive.

You are going to ignore this, by saying that your theory magically only works on large files not small ones, which conviently means that we can't run a test program to demonstrate you are wrong.
But you are wrong.
If two 5 byte combinations give the same hash, then any pair of files which are exactly the same, except that they each start with a different one of those 5 byte combinations, will also give the same hash.
Making the file bigger just stops us demonstrating that you are wrong, it doesn't make you right.

I think it's good for a person to have a problem like this in the background of your brain; if for no other reason than it can be energizing and help you to become a better problem solver.

I myself am stuck on a stabilization system that is probably impossible, but it's a great brain exercise, and not letting go of it helps me to solve problems in the physical world often better than before I got stuck on the idea.

I had previously not heard of Jan Sloot and being that I'm a sucker for a good conspiracy theory and this makes the conversation even more fun for me:)

BTW thanks for supplying those links Moto:

Please read about Information Theory (http://en.wikipedia.org/wiki/Information_theory) and Coding Theory (http://en.wikipedia.org/wiki/Coding_theory).

The ability to compress any given set of data is based on the entropy of the data. Random noise is essentially uncompressible, regardless of what type of compression you try to apply.

I downloaded 1 billion digits of Pi from here: http://stuff.mit.edu/afs/sipb/contrib/pi/
I downloaded the complete works of Gilbert and Sullivan (text format) from here: http://www.gutenberg.org/files/808/

The ~1.3MB file needs just over 100 million digits of Pi to encode.

After byte 1295987, pi position is 103683939
After byte 1295988, pi position is 103684011
After byte 1295989, pi position is 103684099
After byte 1295990, pi position is 103684138

That is almost exactly 80 digits of Pi required for each 1 byte of input files.
(Which is exactly what you would expect.)

So a 100MB file would require 8.4 billion digits of Pi.
A 50GB file would require 4.2 trillion digits of Pi.

The current record for calculating digits of Pi is 10 trillion, and that took 371 days to do, so I think we agree real-time calculation is out of the question?
4.2 trillion digits of Pi require 4.2 trillion bytes to store uncompressed. That is a 4 Terabyte drive just to store Pi to be able to perform your 'compression'.
(Which doesn't work anyway, because it is not reversible.)

We have proven it can't work, repeatedly.
You just don't believe us.

Thanks for your hard work, and this information is invaluable to me.

However, you are still trying to assume I'm going to try and do the entire file in one attempt through Pi, when I already said I was going to limit the size to 500 MB.  But now I see that 100 MB requiring 8.4 billion digits of Pi is far too much, so I must lower my expectations for each chunk.  I will now say each chunk should be 25 MB, which is 1.1 Billion digits of Pi.

We would end up with something like this:

[OPENFILE]
[FileSplitSizeInKB=25600_&_LastChunkSizeInKB=9500]
[Filesize=409600_&_Name="400MBofStuffForWhomever.zip"]
[MChunks=8_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[2,w, xxxxxx, y, zzzz]
[3,w, xxxxxx, y, zzzz]
[4,w, xxxxxx, y, zzzz]
[5,w, xxxxxx, y, zzzz]
[6,w, xxxxxx, y, zzzz]
[7,w, xxxxxx, y, zzzz]
[8,w, xxxxxx, y, zzzz]
[CLOSEFILE]

This chunking method would have an additional obvious benefit of making the MetaData more unique.  But I still have to find a solution as to the problem of having 6 bits being 1s and 10 bits being 0s inside of 16 bit pairs and what that means toward my theory first.  

I do also need to work out a way of creating more uniqueness too, I agree.

This. It seems like you're simply ignoring logical facts. Claiming that people don't understand your idea. Well, we do. Really.

B(asic)Miner, do you even consider at all that it might be possible that your idea is just flawed? As in, plain wrong? Cause if not, no reasoning whatsoever will cure this delusion and we might as well end this discussion, and leave you to your attempts without further hindrance (good luck in that case).

My statement above was more general: "this can happen somewhere therefore you will not get unique results".

However, here is exactly the same issue happening right there in the first few digits of pi.

Maybe by carefully studying this specific example you will discover this is the general problem with the entire idea:


This is a very specific example of exactly what I described as a general problem.

Okay, thanks for your assessment of my situation, all of you.

I just want to know a few more things from you all.  

Let's say I made a modification to my ruleset (actually, it's not a modification, its the ruleset I started with, but felt it would add too much complexity, but perhaps that's what is needed here anyway) so I am going to give the ruleset here again from my original template.  Please someone, the ones who did the proofs before, do a few proofs again to show me my error, or if it's viable now:

Here goes everything:


1) Open file to processed.  Analyze the data.  Then we open our file and begin to record the following:
   A) Original Filename.  Size of the Original File.  Size of Pi Index Used (how big each chunk is to be split into).  Size of the last Mega Chunk in bits (if applicable).
   B) Basekey (the first 64 bytes of the original file, giving the program enough room to establish a unique path given a number of hops.
  
2) Begin reading the data one character at a time (converting hex to Ascii Binary) all in memory using the loaded Pi Index to the Pi Index Size shown in example A above.  Convert everything (all file contents) to Ascii Binary (so every incoming piece of data is exactly 8 bits).  Begin moving forward through Pi by hopping on a single solitary digit called a "Hunt Value" meaning the number we are to be hopping on in Pi.  Starting from the decimal point, begin encoding by hopping.  Start at the 1st 4 in Pi if our first bit is a 0.  If it's a 1, start at the first 5 in Pi.  Hop along Pi, encoding 0s and 1s using these rules.

>>Here comes the rule change!

0 = no change in Hunt Value. 1s = +1 to Hunt Value.   However, how we search for our Hunt Value changes here:   We have to double confirm every Hunt Value against the Index at that location being even (0) or odd (+1).  We can only accept a Hunt Value which double confirms the bit we are encoding.  For example:   Here is  our data to be encoded:

0010 0001  (space added for readability only)

Our first Hunt Value is 4, but we can't stop on just any 4 now, it must be an EVEN 4!  Here goes:   The first 4 in Pi is at index 2 (if you are counting indexes 1-50 and not 0-49 like coders do, so let's please just use 1-50 to keep this easy to read (no offsetting for now))  That means the first 4 IS a 4 AND it's EVEN.  

Let's continue:  the 2nd 0 in our encoding byte example ....  the next four we find in Pi is at Index 19 (odd) but we are looking for an EVEN 4 because our encoding bit is still 0.   Our third 4 is at Index 23 (no good still) so we keep going ....  ah, at Index 36, we have our 2nd EVEN 4!   So now our 2nd bit is encoded ....   Next we are encoding (our third encoding bit) a 1, so now we need to find the next ODD 5 (it increments+1 and because it's a 1, its ODD too) ...  so our first 5 from that location is at Index 48 (an even, so its no good) ... but at Index 51, we have our first ODD 5, we just encoded that bit.  

This process continues accordingly.  This spreads out the hops a great distance, ensuring the path taken is more unique, but at a cost of having to use way more of the Index for every byte "stored" ...

>SNIP


DECODING CHANGES:
We now have an additional checksum for helping to narrow down the possible paths that could have been taken.  

Please someone (murraypaul that means YOU-wink) run the proofs you ran before using this new method and see if they are not all unique. I beg you.  I am very curious to see what comes from this.

That does not fix the problem.

Maybe you're right, but I want to wait until Murraypaul redoes his long list of Indexes again showing that they still end at the same place every time.  Maybe the list will shrink considerably to almost nothing, in which case, I only then need to find a few more criteria to increase uniqueness again.  It may still be possible, but I won't know until the proofs are shown here.  Thanks BurtW.

Generative Art (http://en.wikipedia.org/wiki/Generative_art)
"Generative Art" is often used to refer to computer generated artwork that is algorithmically determined.

Some examples of generative art (5 examples in 451 KB) (http://xchang.in/files/demos.zip)
You can see how much data is algorithimically compressed in these demos (http://en.wikipedia.org/wiki/Demoscene).

More resources
Scene (http://scene.org) 
Assembly (http://assembly.org)
Demojs (http://demojs.org/)

This is indeed impossible, and is simply proven. It bothered me for months about a decade ago when I was working on a brainstorm team to do something very similar, where a small amount of mathematical data could be processed into a much larger file. Herein lies the problem:

In any given amount of bytespace, represented by x as the number of total bytes, you have 256^x different possible combinations.

So, for instance, in a 20 KB compressed file, you have 256^20,000 total possibilities. Assuming that is 98% compression, the original file would be 1 MB. Problem is that a 1 MB file would have 256^1,000,000 different possible outcomes.

This kind of compression does not exist because you cannot use mathematics to create an all-inclusive compression method. That is why most compressions are very specific to the type of file you are working with.

Oh, I love DemoScene!  I was an an Amiga fan in late 80's and early 90's until long after Windows DOS came on the scene, which was totally bankrupt of efficiency compared to the Amiga.  Long live Amiga, forever.  (Okay, fanboy rant over).

I have all the Demoscene demos and have also made a collection of all their music, too, it's some of my favorite music in the world.  


ANSWER:
Yes, to get on with your input, I was trying to think of a way to store a mathemetical version of my compressed key as a jpeg, where each pixel would be black (on white background) to show the movement thru Pi in a stock-exchange sort of drawn squiggly line representing the path taken from all possible paths.  It would draw from left to right when encoding, going from bottom and rising toward the top right, with every 0 going down and every 1 going up and groups of 0s (but not 1s) going level (left toward right).  Then the image would be analyzed and decoded back out using the black bits to help the program to accurately calculate all the ups and down of the timeline, which would be unique.  Because even if the overall file ended up at the same Index, the internal bits would rise and fall differently, creating a key effect from the actual shape of the path taken.  Then you could do optical image recognition on the image, counting pixels, overlaying the computer's approximation until it finally matches 100%, and then you'd know the actual path taken bit for bit, not just because of the end place thru Pi.  It would become a signature based on unique shape (which WOULD BE UNIQUE).  

But I wonder how much data it would take to create such a drawing even for the smallest of files?  The image would become enormous and would have to be saved in a lossless format otherwise if even one bit got changed, it would be a disaster.  For a 50 MB file, it might take a 50 MB image, defeating the whole purpose.  (Although the image would be only in black and white, not color, so that might help some.)  

Just a thought though.  What do the rest of you think of that?

I think it's interesting conversation over a bowl, but not feasible. You're almost guaranteed to end up with a larger file size than you started with by using the methods you suggest.

Dude, you remain ignorant of simple logic, and keep on wandering in meaningless details and experiments. I like your creativity, but there is a much more fundamental problem here, that has nothing to do with Pi, hunt values, flipping bits, decimal encoding paths, or ANY other approach you may think of.

The proof HAS been shown here (numerous times) that it is simply NOT possible, but you choose to ignore that.


Your idea is essentially the same as this: (just replace "2-bit" with "5MB" and "1-bit" with "4KB")


See, you're completely ignoring the actual argument, and keep fiddling with details of an interesting yet inherently impossible algorithm.

If your idea would work, then how about the following approach:

We compress 1280 different files of 5MB each, into 4KB crypto keys. We concatenate all 1280 4KB crypto keys together into one file, which then becomes 5MB in total. We compress that file as well, in 4KB.

So effectively, we have now compressed each 5MB file into 4KB/1280 = about 3.2 bytes. Really? :)

And we could actually do the same thing again, and compress 1280 sets of 1280 5MB files, compress each set into 1280 4KB crypto keys, concatenate the set's crypto keys and compress that to a second crypto key (like above), then concatenate all the 2nd crypto keys and compress the resulting 5MB into a third, final 4KB crypto key. We have now compressed each 5MB file into 0.02 bits.

Really??

^^  pretty darn clear

Hey wait, I've got an idea. Let's compress those 0.02 bits again! Genius!

Eventually we might be able to compress it to zero.  :D

We can store the entire internet in 4KB! ;D (that is including my porn collection... and let me assure you that's HUGE!)

I don't think your poo fetish porn will compress all that well.  :D

Reply by B(asic)Miner:
... but, but, we can just look at all the gazillion possible collisions and pick the one  which contain your porn collection - it will be pretty obvious when you try to look inside it.

The following two letter strings give the same index value as 'Pi', using the new method outlined above:

String: 0M (0011000001001101) Index: 197
String: 0Y (0011000001011001) Index: 197
String: 0e (0011000001100101) Index: 197
String: 0i (0011000001101001) Index: 197
String: 1I (0011000101001001) Index: 197
String: PM (0101000001001101) Index: 197
String: PY (0101000001011001) Index: 197
String: Pe (0101000001100101) Index: 197
String: Pi (0101000001101001) Index: 197
String: QI (0101000101001001) Index: 197
String: rA (0111001001000001) Index: 197

The following single letter strings give the same index as P:
String: 0 (00110000) Index: 110
String: P (01010000) Index: 110

Encoding the same 1.3MB file as before now gives:

After byte 1295988, pi position is 202830581
After byte 1295989, pi position is 202830702
After byte 1295990, pi position is 202830819

So you are now using 157 digits of Pi for each byte encoded, almost exactly the 160 that would be expected.

Encoding all possible one to three byte files using only 62 printable characters gives a total of 426 resulting index values, compared to the 242234 that would be required for full uniqueness.
Of those 426 values, only 26 were uniquely mapped to by a single string.
Lowest used index was 76, highest was 790. (These indexes all start with 1 = the 3 of 3.14).
Most popular index was 517, which was mapped to by 18891 different strings.

Okay guys, no sense beating my head in again and again with iterations of less and less funny jokes.  It's like you're taking your own jokes and compressing them.  And then taking that compressed joke and compressing it again into a smaller, less funny joke.  Until it's you beating my head in for daring to try to imagine something cool.

You win.  I can see the end of the road for this argument.  But I didn't want to give up that easy, you know?  It wouldn't make sense for me to give up that easy after spending 3 years on it.  So you have to understand me at least, right?  Isn't that enough for a tad bit of sympathy.  At least BurtW gave me some love for making him think and he said he enjoyed my thread at the very least.  And I had fun talking with all of you.  Probably the most fun I've had in years back when I first came to China and had like 12 friends and four girlfriends at the same time.  Life was smoking back then, but tapered off pretty rapidly into tedium and boredom.  There are only a few highlights in one's life when you reach over 40 like me.  But anyway, thanks again for all your hard work, and for taking the time to respond to this fanciful imagination.

I will not add more to this thread but I may be back with another idea, but I won't waste your time with that idea until I see how my experiments go.  At this point, I haven't considered that idea long enough to be able to withstand your criticisms of it until I've gone over it all much more deeply.  I think I have another cool way to do compression that will be awesome, but it won't be as awesome as the philosopher's stone I started with.  I guess it wasn't awesome anyway, since it won't work.

But let me ask you all a serious question, and yes, it's related to this very topic which I am closing in the next few posts after your responses end:

Let's say, just for argument, that a way existed to compress huge movies down to 4k or 64k.  That it could be done.  (I'm just saying WHAT IF here, so follow along with me) ...  your approach, indeed all of the most intelligent amongst you's approaches, have been to consider the mathematical probabilities of that plan working.  And if the math says it can't be done, you don't do it.  So let's say a way did exist, and you (because of your adherance to math statistics telling you what can't be done) you never EVEN TRY.  

In sports they say (to encourage athletes who want to give up) that you can only make a basket if you're willing to make a throw.  Am I wrong to try?  Do you think everyone should always listen to mathematical probabilities and not even try?  Because if that were the case, maybe the Universe itself (which is a kind of universal intelligence we are discovering more and more about every day, that is kind of alive in some way, universally sentient) wouldn't be here?  Because the chances of there being life are so small, nearly impossible, that for it to exist is indeed something to ponder deeply.  I'm just saying.  If we never take a shot, we can't ever make a basket.  Don't let math dictate your life, dictate your own life and screw the math is what I say.

The maths are the laws of the universe, and a universal language, but I seriously believe that in 200 years from now (if WW3 doesn't end us "coming soon to a nation near you!") we will have found all the ways to do these same things I'm proposing, because of someone like me didn't listen to the rules and tried it for goddsake.

Thanks for your time.

Is it possible to make a formula to just make a solution set instead of imaging or presenting the binaries?

Instead of file a= 101010001010101010 = 1 * forumula * metadata * index * convert to binary * encode binary ?

However you approach it, it is simply impossible to design a lossless compression algorithm that reduces the size of all possible input files given to it.

Murraypaul, thanks for your hard work.  I am saddened by my failure to make this plan work out.  I don't feel any time was wasted myself, perhaps you do. But I thank you anyway.  You have earned my respect with your diligence and willingness to provide proofs.  I feel I have made some friends here, even if I'm not in your calibre of excellence.  I hope you will want to keep chatting with me down the road.  Please don't take my willingness to try again to find some other solution as a fool-headed waste of time.  It's how I choose to spend my brain power, what there is of it, and what makes me happy.  You can, of course, continue to help me understand my flawwed approaches.  Who knows, maybe I'll actually give one of you an idea of your own.  That would be cool, a payback for your time and effort.  

Peace, brothers.

On the contrary, I think 200 years from now 1+1 will still be 2.


Thanks and sympathy goes to you for starting such an intriguing thread  ;)

Why don't you learn how to program?
This will help you experiment and have fun

I think it would make for a neat 'secret decoder ring' type of encryption, especially for kids learning about maths or computing.
Each byte or byte sequence is assumed to occur in Pi somewhere, so you could encrypt your messages by replacing each byte, pair of bytes, etc... with the corresponding index of that next byte sequence in Pi. Essentially a book cipher, with Pi as the cipher, so no need to share the book.
It will increase, not decrease, the length of your message, but would be a fun computing project.

This.  The concept is called perfect compression* and I pointed that out to the OP.


* Note perfect means all inputs are reduces in size it doesn't necessarily imply a high level of compression.  If you found an algorithm which can reduce all inputs by 0.0000000001% then you have found an impossibility.

Yes, actually, math is even more "universal" as the mathematical laws would be true in any possible universe, even if it were entirely different from the one we live in.
You would really do well to learn how mathematical proofs work, at least as much to understand when someone points out a flaw in an idea that you had. If something has been mathematically proven to be impossible, there is no "try harder" or "maybe in 200 years someone will find a way". It is impossible and it would be impossible in every other universe that could possibly exist.

Onkel Paul

...but each number is stored as an 8 bit character, so you effectively quadrupled the size of it instead of halving it.

While you were mocking me, you inadvertantly summed up my theorim perfectly, making it sound totally awesome.  You have quite a gift for boiling down all my long verbages into one tight, condensed grouping that makes perfect sense.  I can see from this (no joke) that you are more than likely a master programmer with a gift for tight, efficient code.  People who can do this with words can do this even better with coding.

Oh, and it's funny too.  Give the man props!

Oh, duh, I should have seen that one, I would be using characters, which are themselvs ASCII binary, meaning each character would be 8 bits, so I would be adding more data.  So can anyone tell me what is going on with computers that can use something other than binary?  (of course that's not helpful to this discussion, but I am curious).  Is that what quantum computers do, compute with more than binary, but some new base number set?  What exists that would be faster than binary, is there anything?

http://en.wikipedia.org/wiki/Qubit

There are no non-binary computers but in theory you could develop one it wouldn't improve storage though.  Once you abstract everything away it has a physical manifestation (magnetic charge on a disk, high or low signal in ram circuit, transistors in silicon).  Using trinary would be possible but everything would take as much space.  For example most flash ram right now stores 1.5 bits per cell reducing the cell count by 1/3 for a given amount of storage space.

As for quantum computing, no it also works using binary however it involves superposition.  In classical mechanics a bit (or lightswitch if it helps to visualize it) has only two possible states 1=on or 0=off.  Either state can exist but only one state at a time.  In quantum mechanics a qubit can be simultaneously both on and off.  This concept is expanded to larger sets of bits.


http://en.wikipedia.org/wiki/Quantum_computer

Hey guys, you won't believe it,

a computer company is now going live with a version of Jan Sloot's theorum for data compression ... they even have videos to show it is working.

http://jansloot.telcomsoft.nl/     

Click on "Project 7" link (middle left) you will see the video.  This ... this is mind blowing.  I was right.  But maybe not about how MUCH compression total ... it says a Factor of 4, does that mean it's 4 times more efficient?   So a 4 GB file becomes a 1GB file?  Is that how "factor of 4" works?

Let me know what they are doing, if you can find some place to read in English (which I can't yet) ...  

I'm getting a sickening feeling of jealousy here:


It seems this is for real, guys.  Look here on youtube:   http://www.youtube.com/watch?v=ITW5bjTipPA


The video tells some very interesting things:  

1) It won't work with a file smaller than 64 bytes.  Just like I was proposing.
2) At 3:30 seconds in, the video claims "On the Internet, we grab a file that contains random data.  Compression software can not handle such files (sometimes they make these files even bigger). Project 7 does not use any compression techniques and it handles every file as equal, just a file containing a sequence of numbers."
3) It does not compress, it encodes, just like my theory (which you said was impossible!)
4) It's super fast at encoding, just like my idea.
5) It uses 7 layers to cross link the key in a unique way to create encoding like the method I proposed, but using high level maths which I could not propose.
6) The files are stored on your hard drive and can be seen by Windows even though they are now 4-factor smaller.  Movies will play in realtime, but the data is massively smaller on the hard drive.
7) They are working on a 8-factor version but it still isn't working yet.  If they can do 4 and now 8, then at some point they will be doing 16 or 32.  As time goes on, systems become more evolved and get faster.
8) There is one part of the video at 8:30 seconds that says how much encoding was done to the file:   it says original (I can't read it) ... but later it says the video is 7 MB after being encoded and I believe it says the video was 28MB before being encoded.  7 MB isn't as good as my idea, but there is a working version, so it's much better than my idea in reality.  Later, they will have 8-Factor, that means the 28MB video will be just 3.5MB but still be just as high quality (there example is a HD video!)
9) It says it will encode all files equally, something many people here have told me is impossible to say.  One guy here said if anyone claimed they could reduce all files sizes by even 0.000001% they would be lying.  But they demonstrate that they can shrink every file size by the same exact 4-Factoring.  That means you are wrong somehow.

OR ... they are a farce. And this video is fluff, and they are making fools of themselves and advertising something they don't really have.  That would be foolish.  Who knows until it's actually released to the Public?  They might just have a heartattack and all their data cards get lost, and disappear into moon, right?  Hahaha.

Later.


Man!  Here is another youtube:  http://www.youtube.com/watch?feature=player_detailpage&v=kQsWP6n03EU

The Jan Sloot Principal turns out to be none other than the DNA method!  Here is what they have to say on the youtube site video:

"DNA is a different way to store data on a very efficient way like SDCS principle from Jan Sloot. Like SDCS it uses Key-codes (DNA-codes) which are smaller then 257 bytes with a Reference Table.
Goto http://jansloot.telcomsoft.nl/Forum for more information."

I knew it!   A Referencing System!  I am a genius.  But DNA based!   With 3 overlaying streams from different algorythms working in tandem to create a unique key the way DNA does!  DAMN!  OMG!  I should have seen that myself!  3 years and I never thought about DNA, how it works!  This is totally exciting!  

I remember once back in college I was taking an information theory course.  I had stayed out late drinking adult beverages so was kind of hung over.  The teacher was droning on and on about some proof having to do with bases.  You can can create base 2, or base 3 or base 4 and so on devices but what is the mathematically "best" base for information density or some such criteria.

I fell asleep but woke up just in time to see the end of the proof and the answer was e.

Since e is closer to 3 than 2 I expect there is a small amount of gain to be had by doing everything base 3 instead of 2 but base 2 devices are just so much easier to implement.

No.
They might claim they can do it.
That is not, in any way, the same as demonstrating that they can do it.
You have claimed you can compress any file by 90%+. You have not demonstrated that you can do it.

How exactly does DNA store information?  Do you know?

ahem

http://en.wikipedia.org/wiki/Infinite_monkey_theorem

I only read a few posts in this thread and I'm not a "Crypto Compression" expert, but this was my first association.

We tried to tell him numerous times...

Correct. I don't believe it :)

I'll tell you what they're doing: they're lying. Seriously, this is a fraud. 100% sure. Probably to trick venture capitalists into investing money.

You're getting distracted by the presentation, the techno babble, and all the almost-too-good-to-be-true promises. Have you already forgotten about the 2 bit vs 1 bit example? Exactly the same applies here as well.

Let's recap: "make any file smaller by factor 4". If that's possible, i.e. if they can "encode" any file A into another file B, such that B's size is only 1/4 of A's size, they can apply the same method on B as well, thus creating a file C that is 1/16 of A's size. And D which is only 1/64 of A's size. Ad infinitum. See? No matter how fancy their algorithm... IT'S NONSENSE.

In general, claiming that you have an algorithm that can compress or encode or represent or store any file in some smaller form, even if it's just ONE BIT smaller, implicitly (by repeatedly applying the same process on each output file) also means you can store the entire internet in 1 bit. Again, IT. IS. NONSENSE.

Actually, I'll repost my bet to compress any of those 3 files I posted earlier (https://bitcointalk.org/index.php?topic=288152.msg3125723#msg3125723) to the maker of this fraud presentation as well. Compressing or encoding or whatevering any of those three 1MB files into 950K or less = I pay 100BTC. No counter action required in return upon failure.

https://i.imgur.com/BpLHqWg.png https://i.imgur.com/BpLHqWg.png https://i.imgur.com/BpLHqWg.png https://i.imgur.com/BpLHqWg.png (quadruple facepalm)

B(asic)Miner or anyone else, want to do a side bet with me? I'll contact the developer of this DNA / Project 7 bogus. If he accepts my bet, I'll be happy to side bet with other people here on the forum whether he wins or loses. I'll take any stake, small or huge.

you guys did? I did search the thread for "monkey" before I posted though.  ;)


Indeed, this one reminds me of those free energy videos.

that's a good idea. is there anyway you can backup your theory. I'm not a programer however i do believe in possibilities. if you can show me proof of your theory. I'm interested. lets just say that you can provide me with some kind backing to your theory. how much money do you need to get started? any idea on how you'll make your first move? any plans after you find a investor?

It's not possible, and proof has been shown multiple times within this thread. It's simply not a task one can accomplish. Do not invest any money into this idea. OP has no experience programming crypto-compression, and is just shooting ideas around as if he's invented something.

I just had to quote it for posterity.

Given that:

1) you had a military service experience
2) you've reached middle age
3) you have an experience of tilting at windmills with the help of one-man crew

you'll need to:

A) observe where the advice from the school sport coaches doesn't apply
B) switch to the pursuits that are age-appropriate.

The good starting point would definitely be a thorough reading of:

The Ingenious Gentleman Don Quixote of La Mancha

which could possibly help you achieve the peace of mind that you are seeking.

After 15 pages of debunking this nonsense, you come with this? :-\

Ah, yes, the non-binary "bits". Along with the Jan Sloot myth, this had to come up sooner or later. ::)

Look, until here, nobody ever discussed how bits were stored or represented whatsoever. They were considered just an abstract unit of information, cause that's exactly what a bit is: a binary value that can have two states, typically called 0 and 1.

If you can 'encode' single bits as 2,3,4,5,etc then by definition we're not talking about "bits" anymore. And a byte being 8 bit, and a KB being 1024 bytes, this means a "4KB crypto key" is also completely meaningless.

Yes, I can also store a 50 GB movie in only 3 "kilobytes". With the notion that I invented special "bytes" than, due to a new kind of encoding, can actually have 8¹⁷⁵⁴ different values. Brilliant!

Yes, you are. See, in basketball, people are encouraged to make a throw because they could theoretically make the shot. It might be hard, difficult, or almost impossible, but in the right circumstances, with enough effort, and maybe a bit of luck, it could be done. In this compression endeavour, however, you're simply facing a cold, hard, theoretical, fundamental, mathematical, absolute impossibility. You have way more chance of managing to throw your basketball into the sun (or into the Alpha Centauri galaxy, for that matter) than to get this magic compression thing working.

Anyway, as for the movies: yes, some huge movies can probably be compressed to 64K. For example a 3 hour Full HD movie of the inside of a cave at night, i.e. total darkness. But in general, no.

Same argument that has been repeated over and over: if you can compress a huge movie to 64K, then you can also combine 3000 of such 64K files (each representing a single huge movie) into one new movie. Just displaying every 64K in sequential frames (movie shots), representing the bits in black & white or RGB pixels or whatever. It will probably look like a strange static noise movie, but it's a movie nonetheless. Now we can compress this new movie into 64K as well. So, if we can compress any single huge movie into 64K, we can also compress any 3000 huge movies into 64K. And also 1850000000 huge movies for that matter. Logic impossibility.

thanks for the tip. however i am asking for some kind of proof showing me that he can back his theory up. i am not going to invest if he can't come up with his own proof. and no, someone else video isn't going to pass. he did state that he has already completed his program of compression. all he needs to do is show me.

i'm not a computer programer or anything of that nature. all he needs is to show me his own proof. something he hasn't done through out this entire post. i'm not trying to doubt the man. surely he can provide me with some type of preview of his completed work. not just something that someone else had claimed or so called completed through theory. again it is only theory. maybe he can make us a video and post it on youtube so we call can see it for ourselves.

But.. solid 100% proof that it can't be done has been posted numerous times.

Seriously. It's not a matter of some smart algorithm that we didn't consider, some new fancy approach, or 'thinking out of the box'. The idea he proposed (as well as the video from that other site) is simply a logical impossibility. 

so in other words... it is just a roody poo idea

Correct.

It's like saying "a found a really special magic number that is even, yet it cannot be divided by 2!" but more complicated, so its logical inconsistency is less obvious.

I feel bad for B(asic)Miner though, cause I'm sure he genuinely believed he found something marvelous. Contrary to the maker of that Project 7 / DNA video who is definitely a fraud.

Proof or it did not happen, and will not happen.

Just publish it. You may not die rich, but you will live for all eternity in the science books.

If this works, no need to teleport people. Just get a loyal soldier, and create a clone army. Then do world domination. Or duplicate gold, because you can't double spend bitcoins, but you can "teleport" gold back and forth.

I'm sure his magic compression tech can triple spend BTC :-)

I actually had almost this same idea as OP did, about 15 years ago. About the time I designed a steam-engine from scratch (then discover that it had already been invented). After thinking about it, I realized it could never work, at least not the way that I understood it.

Now I have this perpetual motion machine design, which might work. Then I realized, it's not really independent, and relies on the planet for power, or the other version requires very high quality mechanical parts. Or if it really worked, it would break some laws of physics. But I never had the funding to try it, since it would be expensive to build. So I might as well just use wind, solar or something and make those more efficient.

But just for fun, try grabbing about 10 UPS (uninterrupted power supplies), charge them all, then plug them in series, then plug the last one to the first one. Woot! It will stay on for about a few months, then eventually die. And if you use power from any of them, they will all eventually drain faster. You only need 3 to make this work.

Or I had this solar power generator once, then I put the lamp that it powered over it. It got dimmer and dimmer. hehe.

Sounds like you and OP would get along just dandy ;)

lets all teleport back with our Asic and start mining btc too.

He wants to write a book just referencing other words from other sources. The problem is that he thinks the references wont take as much space as the words themself.

It leads me to think about the .par file system they used to have on the usenet groups. http://en.wikipedia.org/wiki/Parchive

I don't think what OP wants is possible. So while he has ideas, I'm not sure I can get along with him. He'd be rattling off too much and I'd just be annoyed.


There's no mention about time travel here. But I'd take a USB miner with me. Small. Concealable. 2.5 GH/s in 2009? ROI overnight I think.

..what happened to OP? Did he give up on his billion-dollar idea?

He should give it away for free... You know, like Volvo gave away the 3 point seat belt to all car manufacturers instead of patenting it.

What happen to op? Did he finished this project? I doubt if this really possible.

OP realized his idea is not possible, thats why the silence  ;)

I obviously didn't take the OP seriously. That said, some advanced theories behind LOSSY compression are interesting. For example, AAC-HE, fractal compression in JPEG2000 or procedural generation of textures in games (remember the 96kb 3d game?).

Bringing a 2.5 GH/s miner to 2009? First, you should bring a recent cgminer too. Secondly, you would just kill the network.

You right, all you need is bring cgminer and buy a GPU :)
And sell at 250+ USD/BTC
And profit  :o

Software and hardware, goes without saying. And I wouldn't kill it. I'd do just a block or two, then stop, or wait 6 hours. Or throttle it. I'll get a block a day or something.

Knowing the future of 2013, I don't need to get greedy. Hell, if my miner gets busted after a block, it already ROI'd.

That's a great idea Dabs. Now let's get the OP to create a new groundbreaking technology for time travel.

If it's really time travel, bringing any information back is going to make you rich. Winning numbers, stock performance, buying into Asicminer, ... Trying to eat the same taco... Oh wait, wrong game. That's Plants versus Zombies 2.

You don't nee,d a GPU or cgminer.  Using the default miner built into the Satoshi client a high end CPU could mine 10-20 blocks per day.  For the first 6 months the average hashrate was <10 PCs and the next six months weren't much more.

If you had a computer, any computer you probably would get 5% to 10% of the blocks for the year.

Yeah... These alt coins look interesting. If only they survive long enough.

Premining. ::)

Would you be trying to eat the same taco, or would you need to? For the greater good of the universe

No, I am still here.  A while ago this whole board got crashed when some kind of a hack happened and I was devastated that my whole thread was lost.  I couldn't find my thread at all, whole whole site was down.  Recently, while talking to some others about this wonderful but lost thread where I got schooled by all of you for trying to imagine something awesome, I did an internet search and by coincidence this very thread showed up in the search results!  I couldn't believe it when I opened the thread and found this website back and my thread back.  Now I'm copying out all of this text and saving it for future reference (well, except for the many many flames I got, who needs to remember that!?)  Your explanations were quite invaluable to me in understanding.

However, some of you are just too ardent in believing that this is impossible and cannot be done, and I still believe that encoding can, if done correctly, result in a significantly smaller filesize than the original file.  And I've devised a method of showing it might work.   Essentially, it works like this:

Layer 0:
Input Any File and Convert whatever language into binary.  Now grab the binary data in chunks of 4.

Layers 1 & 2:
http://imageshack.com/a/img46/6823/b4g0.jpg
Convert Using the Table Above.  Initially, yes, this does double the size of the file (since we'll be taking ONE 8-bit character and turning it into TWO 8-bit characters), but it must be done to begin the process of formatting into a recognizable structure.  Each layer has its own engine and rules that must be followed.

All Binary is converted into Ascii characters (inside a text document), letters small-a "a" to Large H - "H"  aAbBcCdDeEfFgGhH for layer 1.  Now ALL of the Binary data is converted into letters a to H (small and Large) characters.  And that is saved into a text file.

Now in Layer 2 (which is actually a group of sub-layers needed to convert all the Layer 1 "text" data) out of Layer 1 form into Layer 2 Form, so all the data is in an entirely new form, small i "i" to Large Z going  iIjJkK .. to .. zZ.  Its too hard to explain it all right here, so I'll move on for now, but this process is rather complex and lengthy, but must be done before going on to Layer 3.

Layer 3:  Another Grouping of Layers dedicated to changing all the i-Z data from Layer 2 into the numbers and symbols used in Ascii.
http://imageshack.com/a/img189/6539/9j3f.jpg
This complex system slowly painstakingly changes all Layer 2 data into Layer 3 data, where now all the data is now symbols and numbers.

Now at this point, you have 3-point system for changing all the data interchangeably back and forth, but what does that do?  How can that encode (and shrink data)?

Here is how:  Imagine a Crossword Puzzle 20 spaces long, where the previous data sets were sorted into the cells,  so that every 21st index forward (drops down and back to the extreme left cell) and is now under the space above it.  So every 21st space, you have a new row.  Now, the software looks for patterns according to the Layer Tables I've drawn out above.  They are not complete tables, as I've said the tables are multi-layered to account for all of the variables in the data sets.  Let's say you had this:

1      2      3      4      5       6      7      8       9     10    11    12     13    14    15    16    17    18   19   20
E   g   A   D   f   D   B   g   H   a   c    A   C   d   c      F     d      E     H    A
D   B   b    C   F   g   d   C   d   a   F   a   a   C   F      c     c      B     c     h
f   b   a   D   A   H   g   h   h   C   b   H   d   c      C      d     G     F      f     E     

3 Rows of 20 Columns.  Look at Row 1 and Column 10, and then look below it at Row 2 Column 20.  Both are "small a"s.  They can be encoded by exchanging Row 1 Column 10 with a reference from Table 2 above.  See the Table 2, where it says "aa" and below it there is a "i" ?   So then this happens:

1      2      3      4      5       6      7      8       9     10    11    12     13    14    15    16    17    18   19   20
E   g   A   D   f   D   B   g   H    i   c    A   C   d      c      F     d      E     H    A
B   b    C   F   g   d   C   d   F   F   a   a   C   F      c      c      B     c     h     f
b   a   D   A   H   g   h   h   C   b   H   d   c      C      d     G      F      f     E

The 2nd row "a" below the new "i" above it is deleted, and the entire sequence proceeding it is shifted left one space through the entire table.
When all of the data that can be encoded is removed, you begin to see the compression (or encoding whatever you would call it).

The whole thing is based on the Ascii Character Table, which could be used to reference itself.    When the program is going in reverse mode (extracting out the data) it comes to the "i" in Row 1 Column 10 and sees the "i" as "aa" (one "a" in the space where the "i" is and one "a" in Row 2 Column 20, directly under it.  First it shifts the whole table forward by one to make a space for the "a" to be added back in there."

This is a really tiny explanation, but it should be enough for the majority of you to see if this logic is feasible.  If you are changing two pieces of data into one based on how it spatially aligns in a crossword puzzle, then you can how using spatial alignments is how you can throw out one piece of data and yet still be able to recover it.

It's like having a bunch of people, with random normal names, stand in a line.  And then you say if there are two "Jim's" standing in the crowd, we can take them both out and replace them with a "Jimovious" a name no one else will have in that group.  Now we have shrunken the number of bodies in the crowd by 1.  But if asked to reassemble the original crowd, we just say throw Jimovious out and replace with 2 Jim's.  The engine knows where to put them in the crossword puzzle, because it would not have encoded them in the first place unless they had aligned directly above and atop each other.  So the engine knows to read two lines at a time, comparing  IF Row1Caret=Row2Caret then DO REPLACER().    Something like that.  Do you get the idea? 

Please friends ...  let me know your thoughts.

Oh dear.
You clearly haven't understood the basic point, which is that if you have X total number of input files, each of Y bytes size, then the total size of all of the encoded files must be at least X*Y.
If it is not, then you have lost some data, and two of the encoded files will map back to the same input file.
It doesn't matter what process you come up with or how clever it is, the basic fact is that for a compression system to be lossless, information cannot be lost.


And you have inadvertently undermined your own point.
"Jimovious" takes more space than "Jim" twice.

Sorry, but the data in my post got shifted and garbled ... so I made another image.  Sorry the others were so big ... this one is a bit smaller.  Now you can see what I was trying to say in the Row-Column part in the above post ...  here it is:

http://imageshack.com/a/img30/3839/tzya.jpg

OP just let it go mate, move on.

Patent / Open Source or it didnt happen.

What you describe sounds like a very primitive compression method with a bit of "the engine just knows" magic.

I'll tell you what, I will decide when to quit.  And you can decide when to quit telling me to quit.  And we will see who quits first.  Deal?

Here is the whole thing boiled down a succinct as I can make it.   There's no magic here. 

Import data, convert to binary.  Save.  Re-import binary, convert to Layer 1, letters a-H (aAbBcC... etc) according to the Table above.  The file will double in size.  Save.  Re-import and begin Layer 2.  Encoding will begins here.  Encode 1st chunk of 1024K using Crossword Grid alignment, 20 spaces per row.  Fill Array with data from Layer 1.  Now search data from current row, 2 rows at a time simultaneously, 20 spaces apart (essentially on top of each other in the crossword puzzle grid.  Let's call that the CWG from now on.  When a match is found, replace topmost row's match with unique identifier that means the same thing to the 2nd Layering engine as the match found.  For example "aa" is replaced by "i"  Now delete the bottommost match, leaving an empty space (the compression/encoding).  Now shift the whole array to the right, displacing that empty space, which now becomes a zero at Row 1 Column 1.  Now continue forward.  Find all matches until the last line is (the last 20 cells are) reached.  At the last line, no more compression can be done because there isn't a line under it to compare to.  What we are left with here is essentially a boiled down key.  Nothing more can be done with this chunk.  That key is saved inside the file we are building as such:

1004(0)_GbcDeEafFBAAcbeEBDfga_6(0)

Where the first block above is the topmost last line to the halfway point of the line and the 2ndmost line is the bottommost line to where it ends halfway (20 spaces total combined) and the final part is a message to software that 1004 zeros precede those two keys.  Now the engine can get rid of the 0's, leaving that small chunk to retrieve all the data later.

It would do so like this .... 

0000000000000000000
0000000000000000000
0000000000000000000
0000000000000000000
0000000000000000000
0000000000000000000
GbcDeEafFBAcbeEBDfga000000

Now the engine counts how many zeros are in the block before the first actual piece of data to know how many iterations it had done to reach that final sequence.  It counts the zeros, here we see 120 zeros total.  The engine is told the key goes on the last line plus the number of zeros (empty spaces) that were left over. Now having figured out how may iterations to start from backwards, it begins comparing data back out, starting by re-ordering the entire sequence to the left.  So the key would actually look like this:

0000000000000000000
0000000000000000000
0000000000000000000
0000000000000000000
0000000000000000000
0000000000000GbcDeE
afFBAcbeEBDfga000000

Which we can easily see the last line because the G and the a are not overtop of each other, meaning this is indeed the true last line.  Depending on how good the compression is, the last line can occur anywhere in the block, as such :

0000000000000000000
0000000000000000000
0000000GbcDeEafFBAc
beEBDfga00000000000
0000000000000000000
0000000000000000000
0000000000000000000

As long as the block is totally intact and none of the pieces overlap, it is complete.  The reason we need to know how many empty spaces were left at the end is so we can separate how many iterations occured with how many empty spaces were left, since not all the zeros here mean iterations. 

I hope you can see this as clearly as I see it in my head.  Its efficient and would totally work.  I hope you will be able to see that by studying this.

So you invented the Tardis from dr who ? lol

It looks like you are still having a lot of fun thinking about this.  Good for you.

If you have X possible input characters then you have X^2 possible combinations.
So you have reduced the number of characters, but increased the storage required for each one.
There is no overall saving.

You need to accept the basic problem:
X distinct input files must generate X distinct output files, otherwise you cannot retrieve the original files.
Do you agree or disagree with this?

I've thought about similar methods of compression... In the end, the issue isn't whether you can compress data down to almost nothing, its how much time it takes to decompress the data. Compressing data down is only part of the problem.

I don't believe that I am able, at this time, to accept your basic problem (premise).  I would be like the Wright Brothers agreeing with people who said flight was altogether an impossible thing meant for birds and if God had wanted Man to fly, he would have given him wings.  

If I were to accept your basic premise, I would have to say that one word cannot possibly mean anything more than just one thing.  Therefore, the word "bow" cannot mean to bend over at the waist, when it clearly means a piece of ceremonial cloth tied at the neck.  And there goes the other half of the "____ and arrows" equation.  One input can in fact stand in place of many outputs, its all based on relative meaning.  Space.  Whether that space is in our imaginations or in the real world.  We can clearly simultaneously understand that "bow" means bow and arrow, bow tie, and take a bow.  One input many outputs.  There are millions of guys named "Smith"  ... by your logic, there cannot be more than one Smith, since 1 input = 1 possible output.  

By using tables, one can generate meanings for given sets of data.  I can say in my Table that "aa" = "i"  ... now let's say there are 50,000 matches in a 100,000-text piece of data.  That implies 100,000 total pieces of data that can be compared, being reduced from 2 pieces to 1 piece.  I save the data out using software based on this system, then it's now half the size it once was.   The filesize is now 50,000 characters in bytes (whatever that amounts to) but was 100,000 characters in bytes.  How can you tell me that it was not cut in half by my method using logic, when its now clearly half the size it was (hypothetically)!?

You say we cannot take 2 bits and represent them with only 1 bit.  Perhaps that's not true.  As things stand, a bit is merely a single register either on or off.  But some hardcore engineers have demonstrated that a gate can float halfway between open and closed, creating a third state, a fuzzy state.  So what if we designed a voltage subroutine into the programming of the compression/encoding scheme that was able to read the voltage level of a bit?  Now we see a bit can float more to the left or more to the right.  Something like this:   0    |  /  1   being 30/70    or    0  \ |   1  being 70/30.   So now we can say If the 2 bits are 00, they are 100% right.  But the bits are 01 they are 30/70% in favor of the right.   And if they are 10 they are 70/30% in favor of the left.  And if they 11 they are 100% left.  Thus, that one bit now has 4 states to represent 2 bits in place of 1 bit.  

Its still one bit, it still only holds one bit of info.  But we found a way to read that bit differently (bow-arrow, bow tie, take a bow, bow of the ship) despite the bit holding no extra space.  We used another method to change what the bit could do.  We created a reference where no reference previously existed.  That's all I'm trying to do with this new theory.

I also come up with a solution for the decompression time, so here is my solution and the next problem.

This may not be the best explanation, but I'll try. Imagine a user downloads the tiny compressed file that represents 10 gb of data. Now imagine writing that 10 gb of data to a hard drive hundreds of times in order to get the true 10 gb of data. During that time your using full resources of your computer or device, rendering it somewhat useless.

So of course I couldn't get that far and not come up with a solution. So what if in the new world we had devices that intercept the information and decompress 1 layer at a time, and each layer of decompression streamed it to the next layer of decompression. Now we are no longer compressing files, but internet data as it enters your home. Making it possible to download the 10gb file as if its much smaller.

Well, the problem here is... The tech has already been invented and utilized.

You are almost there.
What you have shown is that "Smith" is not a unique compression output for a person whose surname is Smith, because many many people would also be compressed to the same name. So Person->Surname is a lossy compression method, because it is impossible to retrieve the original person from just the surname.
Similarly, natural language is a lossy compression of the objects or ideas it represents. Given just the word "bow", I cannot know if you meant bow and arrow or take a bow.
Lossy compression schemes can reduce sizes for all input files, but it doing so they lose the ability to exactly reconstruct the input file from the output file.
MP3/AAC for music and XVID/MP4 for video are common examples of lossy compression schemes.

What you have claimed to be able to do is create a lossless compression scheme that can compress [transform to less than their original size] all input files. That is simply not possible.

There are 256 possible different one byte input files.
If there were less than 256 different output files, then two input files would be mapped to the same output file, and it would not be possible when decompressing that output file to know which of the two input files was meant. Information would have been lost, so this would be a lossy compression scheme.
So our 256 input files must map to 256 different output files. Each file must be at least one byte long, so the total size of all files must be at least 256 bytes, so no space has been saved.

There are 256x256 possible different two byte input files.
If one of them was to map to a single byte output file, it would have to be the same as one of the output files created by compressing one of our single byte input files, which would meant that we could not differentiate between those two files when decompressing, so this would be a lossy compression scheme again.
So there must be 256x256 output files, each of which is at least two byes long. So no space has been saved.

By induction, the same proof shows that for any input file size, the total set of all files of that size or smaller must map to a set of files of at least the same total size. Hence the average compressed size of any one file must be at least as large as the size of the file itself.

I thought about this, or very similar to this, several years back. I could never get it to work after throwing everything at it. So I put it aside and worked on solving Fermat's last theorem. When I was close to a solution, someone beat me and published it.

I went back to my ultimate compression algorithm, and the best I could come up with that actually worked was fractal related and only on specific kinds of data.

I don't want to discourage you, but so far, your table method does not make sense to me, or I can't see how it can work, at least on normal data.

Try it on the blockchain and see if you can losslessly compress 14 gigabytes to something significantly smaller. Give it away for free. You'll get rich somehow.

He will be enriched by the experience!

comments inside the quote marked with >

Edit:
Fun read for those who want to know more about our mystery man:
http://encode.ru/threads/1789-100-Lossless-Compression-Theory-Please-Join-Me
he also looked for help from people who know their 1:1 of encoding


Edit2:

http://ticc.uvt.nl/~pspronck/sloot.html

by now I have to thank you for posting this. Quite a lot of fun reading thanks to it :D

Not sure if this is serious or a troll but you sure are presistent.

http://en.wikipedia.org/wiki/Pigeonhole_principle
http://en.wikipedia.org/wiki/Lossless_data_compression