A new generation of encryption that would change the world... and bitcoin

[Glimanas]

What if it would be possible to create a new kind of encryption that it would basically create a hash for a file of any size and the transfer of this file would just be done by sending the hash?

You would be able to send TBs of data instantaneously.

In the bitcoin world this would be revolutionary in many ways, if you look how long you have to wait in case you want to sync a bitcoin wallet from scratch...

In practical terms, this magic algorithm would create an hash for the file byte stream, let's say for a file of 1 TB of size, you would then send this hash to someone, the hash would be inserted in an app that would decrypt it and basically the file byte stream would be created right there, locally on the receivers pc.

Any ideas if this is actually possible or if there is something out there or someone working on something similar?

[DannyHamilton]

Yes.

Compression already exists.  It has existed for many decades.

Compression can reduce information to its minimum essential storage.  However, to compress information beyond its minimum essential storage would require losing some of the information (that's what jpeg compression does, it throws out information that it decides isn't typically "important").

Newer and better compression techniques will continue to be devised, but you're never going to store TBs of actual information in kilobytes of storage.

You may as well be asking:

"What if it would be possible to create a new kind of travel that would basically create a way to travel backwards in time to early 2009? You would be able to mine entire blocks of 50 BTC every 10 minutes with your desktop computer. In the bitcoin world this would be revolutionary in many ways, if you look at how long you have to wait if you want to mine a block today. In practical terms this travel would move you backwards through time instead of space, and basically you would be able to go back to 2009.

Any ideas if this is actually possible or if there is something out there or someone working on something similar?"

[Glimanas]

Newer and better compression techniques will continue to be devised, but you're never going to store TBs of actual information in kilobytes of storage.

Sure about that? I don't really like the word "never".

What if you can compress TBs of data to something like this:

d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f

If everyone would always say never and actually do nothing about ideas you would not be sitting today in the 21st Century in front of a computer typing some text and sending it across the world in a fraction of a second.

[achow101]

Newer and better compression techniques will continue to be devised, but you're never going to store TBs of actual information in kilobytes of storage.

Sure about that? I don't really like the word "never".

What if you can compress TBs of data to something like this:

d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f

If everyone would always say never and actually do nothing about ideas you would not be sitting today in the 21st Century in from of a computer typing some text and sending it across the world in a fraction of a second.

First of all, that string is not kilobytes, that is 29 bytes of data, and that is way too small. Sure there may be newer and better algorithms to keep compressing data, but there is a limit. You will lose too much data so that when you decompress it, the data isn't complete and becomes unusable. It may happen and be able to compress TBs of data to Mb or even Kb, but it probably isn't happening in your lifetime.

[OROBTC]

...

I am winging this here, and no doubt missing some relevant technology, but the below *seems* possible:


1)  You have a file of 200 photos you would like to send (say of chip designs to an Intel or someone).

2)  With a new publicly available program and hashing function, you hash that file to something like:

     d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f

3)  You put a password on your hash function, so that the intended receiver can enter the hash no. and password into that program

4)  The program* then crunches out the original file (photos in this case).


* One thing (among many other things) I do not know is if hashing functions can be run in reverse by a program...


EDIT: Maybe the hashing function could be modified so that extra characters would give the program additional instructions in how to reconstruct the original files... (?)

[achow101]

...

I am winging this here, and no doubt missing some relevant technology, but the below *seems* possible:

1)  You have a file of 200 photos you would like to send (say of chip designs to an Intel or someone).

2)  With a new publicly available program and hashing function, you hash that file to something like:

     d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f

3)  You put a password on your hash function, so that the intended receiver can enter the hash no. and password into that program

4)  The program* then crunches out the original file (photos in this case).


* One thing (among many other things) I do not know is if hashing functions can be run in reverse by a program...

Hashes are one way functions. They cannot be reversed.

[Glimanas]

...

I am winging this here, and no doubt missing some relevant technology, but the below *seems* possible:


1)  You have a file of 200 photos you would like to send (say of chip designs to an Intel or someone).

2)  With a new publicly available program and hashing function, you hash that file to something like:

     d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f

3)  You put a password on your hash function, so that the intended receiver can enter the hash no. and password into that program

4)  The program* then crunches out the original file (photos in this case).


* One thing (among many other things) I do not know is if hashing functions can be run in reverse by a program...


EDIT: Maybe the hashing function could be modified so that extra characters would give the program additional instructions in how to reconstruct the original files... (?)

You got the idea.

However of course this could not be done with a 1 way encryption method like SHA. It had to be with some kind of new 2 way encryption method.

I think there is room to evolve here, and maybe doing such an algorithm is not a hard as it looks.

Just a quick and dirty example:

Lets say this new algorithm would simply remove some kind of byte stream repetitions and would just store the position of those removed repetitions in the end of the byte stream.

You would probably save some bytes... then you do the same for X other amount of repetitions and you just store in the end of the byte stream the positions where the bytes were removed.

The decryption app would add those removed repetitions by reading the positions in the end of the byte stream.

[CIYAM]

Seriously I am surprised the OP doesn't have an ad sig to be posting such nonsense.

You can't compress random data. Does that make sense to you or not?

(next step is to get yourself an ad sig so you can make money with such posts)

[Lauda]

Newer and better compression techniques will continue to be devised, but you're never going to store TBs of actual information in kilobytes of storage.

Sure about that? I don't really like the word "never".

What if you can compress TBs of data to something like this:
d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f

He's correct. If you were a hardware or software engineer you would know the limits. The thread has been mostly answered by Danny. Better encryption is going to be developed; algorithms that can be used with Bitcoin and ones that can't. However, you are never going to be able to store wast amount of actual data in a tiny compressed format. Even if that was possible, you would be decompressing for hours which would make it redundant then.

[Glimanas]

Seriously I am surprised the OP doesn't have an ad sig to be posting such nonsense.

You can't compress random data. Does that make sense to you or not?

(next step is to get yourself an ad sig so you can make money with such posts)

Random data? what are you talking about? I think you didn't read my post.

"Can't" and "never" are not exactly the right words when it gets to evolution.

[eli113]

actually what OP said is posible but not efficient ...

in order to compress sequences of raw data you may use a dictionary like table and address to key numbers (hashes)
parts of those sequences , then is posible to present a little file with reference to all data ...

but then a new problem arises , tables for such references will be bigger than the original file !!!
if sender and receiver already have the dictionary then they are able to transfer those compressed files

programs available in the market doing that kind of 'magik' to compress with good ratios already.
some sequences are frequent in text,images and such , those programs have a well-tuned dictionary
in order to compress more those programs will need even TB of help dictionary files ROFL

happy holidays to all and best wishes for 2016 as well

** but title is misleading - encryption actually making more data and save nothing but provides safety.

[CIYAM]

"Can't" and "never" are not exactly the right words when it gets to evolution.

I think you should watch the YouTube video about the "expert" and the seven perpendicular red lines (your idea makes about just as much sense as that does).

It is a bit like trying to explain to people that private keys are *safe* when they are created randomly. They will always respond with "but there is a chance I might create the same key as someone else?".

So yes - there is a chance you could do that and there is also a chance that there is a bird about 1km above your head right now that is going to drop a shit that will hit you (but I doubt you are going to now move because I have stated that are you?).

[shorena]

Just to clearify we are not talking about encryption here as the title suggest, but about compression or information encoding.

actually what OP said is posible but not efficient ...

in order to compress sequences of raw data you may use a dictionary like table and address to key numbers (hashes)
parts of those sequences , then is posible to present a little file with reference to all data ...

Yes, but this limits you to the number of entires in your table. It does not allow you to transfer arbitrary data. You can say lookup entry x, but if there is no existing entry for what you want to transfer you cant use the scheme.

but then a new problem arises , tables for such references will be bigger than the original file !!!
if sender and receiver already have the dictionary then they are able to transfer those compressed files

They dont actually transfer the compressed files, they just transfer a pointer. You cant call "look up shanon entropy on wikipedia" a compression of the actual wikipedia article[1]. Its just a reference that allows you to access the same data I accessed, because you understand the reference "wikipedia" as a specific homepage, you assume the language english and you understand how to search for specific information on that homepage.

programs available in the market doing that kind of 'magik' to compress with good ratios already.
some sequences are frequent in text,images and such , those programs have a well-tuned dictionary
in order to compress more those programs will need even TB of help dictionary files ROFL

Constructions of huffmann trees are common yes. They are limited though. As I said above if you want to be able to transfer arbitraty data and not pointers you need to be able to encode everything. Encoding everything is not possible with a limited message length. Error correction is also a thing on the fundamental level of data transportation, which adds more data.

happy holidays to all and best wishes for 2016 as well

** but title is misleading - encryption actually making more data and save nothing but provides safety.

[1] https://en.wikipedia.org/wiki/Entropy_(information_theory)

[Drewski]

I'd like to know where this idea of magic algorithm comes from.  Typical compression works like a dictionary.  The program looks for repeats in the file, and then designates a shorter abbreviation for that chain.  So a 128 bit chain that recurs in a file can be designated as an 8 bit chain  You still have to send the original 128 bit chain once, but every time after that, you only have to send the 8 bit chain.

[CreativeCarol]

Normally, Bitcoin is able to be utilized and manipulated to find very many different routes for new generations of encryption I believe.

[Eric Cartman]

Impossible.

Search about hash collision, how would you deal with them?

[franky1]

Newer and better compression techniques will continue to be devised, but you're never going to store TBs of actual information in kilobytes of storage.

Sure about that? I don't really like the word "never".

What if you can compress TBs of data to something like this:

d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f

If everyone would always say never and actually do nothing about ideas you would not be sitting today in the 21st Century in front of a computer typing some text and sending it across the world in a fraction of a second.

What if... you, yes YOU could manage to do it without any data loss/leakage/corruption

then YOU will be the inventor of something worth billions to the corporate world
so because you dont like the word 'never' i think you should get started making your dream a reality
and please don't reply that you 'never' programmed before. or that you think you will 'never' program the solution in the next 10 minutes..

so i look forward to your source code in 9minutes 55seconds

[Amph]

Impossible.

Search about hash collision, how would you deal with them?

they are basically a non-issue, and there is no way to know if one address already collided