Is my method a secure way to gather entropy?

[profitgenerator212]

I have invented a new way of gathering entropy, in order to generate keys from it, I it all described here in detail:

https://steemit.com/security/@profitgenerator/generate-secure-random-passwords-or-private-keys

In summary the method is this:

• Use the internet to find public data on it that has large information content: files,music, videos, etc.. , or add your own if possible
• All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source
• We gather X number files from a search engine using random keywords, we gather another P number files , where X = P
• Instead of using XOR, due to fears of short key, we just encrypt all the P files with AES256CBC with separate keys generated with the OS's default RNG, the AES256CBC encrypted files will be E
• Then we take the 512 bit hash of the E files, add a random salt to each of them, and use these as separate AES256CBC encryption keys with which we encrypt all the X files separately with AES256CBC
• Then all encrypted X file are already a unique good source of randomness, then we put all X files in a .tar file, and this is our MAINENTROPY.tar file

So when we just want to create a password, we just take the SHA1,SHA256, or SHA512 bit hash of the MAINENTROPY.tar file and that will be our private key.

After use, we just add another random file inside the archive, to act as a salt or nonce, and produce a new unique hash next time we use it.


What do you think?

[cr1776]

"Use the internet to find public data" - using TOR?  New sessions each time?  (to avoid someone monitoring your searches or your ISP monitoring your downloads)?   Security through obscurity is worrisome.

Seems complicated for the benefits.  

I have invented a new way of gathering entropy, in order to generate keys from it, I it all described here in detail:

https://steemit.com/security/@profitgenerator/generate-secure-random-passwords-or-private-keys

In summary the method is this:

• Use the internet to find public data on it that has large information content: files,music, videos, etc.. , or add your own if possible
• All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source
• We gather X number files from a search engine using random keywords, we gather another P number files , where X = P
• Instead of using XOR, due to fears of short key, we just encrypt all the P files with AES256CBC with separate keys generated with the OS's default RNG, the AES256CBC encrypted files will be E
• Then we take the 512 bit hash of the E files, add a random salt to each of them, and use these as separate AES256CBC encryption keys with which we encrypt all the X files separately with AES256CBC
• Then all encrypted X file are already a unique good source of randomness, then we put all X files in a .tar file, and this is our MAINENTROPY.tar file

So when we just want to create a password, we just take the SHA1,SHA256, or SHA512 bit hash of the MAINENTROPY.tar file and that will be our private key.

After use, we just add another random file inside the archive, to act as a salt or nonce, and produce a new unique hash next time we use it.


What do you think?

[piotr_n]

you're overcomplicaing it, imho

you'd be better off just taking photos or recording voice, instead of downloading existing media files from internet.

each analog-to-digital converter adds some noise to the output data.
with a proper application, it can actually be used as a pretty perfect random number source.

[BurtW]

All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source

This is a red flag for any security related algorithm or methodology.

[profitgenerator212]

"Use the internet to find public data" - using TOR?  New sessions each time?  (to avoid someone monitoring your searches or your ISP monitoring your downloads)?   Security through obscurity is worrisome.

Seems complicated for the benefits.  

Not necessarly, because the data is already out there, so public data is already public data.Just like if you are english speaking, and your mnemonic phrase consists of english words, the attacker won't try to brute force chinese words.

The same way even though by downloading the files over a monitored internet connection, a targeted attacker could know what you are up to, but still can't guess the final entropy.

Or in other words, the haystack gets smaller, but it's still big enough to provide good security in my opinion.

you're overcomplicaing it, imho

you'd be better off just taking photos or recording voice, instead of downloading existing media files from internet.

each analog-to-digital converter adds some noise to the output data.
with a proper application, it can actually be used as a pretty perfect random number source.

That is almost the same as downloading from the internet.

If you go out in your city, and start photographing buildings, that is exactly the same thing as downloading over the internet, minus the fact that a targeted attacker doesn't know what you will do with the photos, but the photos themselves are already public, because anyone else can take them (with smaller or larger variations of angle, brightness, etc...)

But a phrase might not be unique enough, so we need to combine multiple sources of uniqueness.

All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source

This is a red flag for any security related algorithm or methodology.

What are you talking about it? We just keep the entropy secret, and all materials that we have used, will be deleted afterwards.

It must be kept secret otherwise it's not unique enough, and others can guess your passwords too.

[piotr_n]

No it isn't the same thing.

When you make a photo yourself, nobody else has its copy.

You asked about 'gathering entropy'.
There is no entropy in files that you download from Internet.
You're not gathering entropy, but just obfuscating the origin of your seed.

[profitgenerator212]

No it isn't the same thing.

When you make a photo yourself, nobody else has its copy.

You asked about 'gathering entropy'.
There is no entropy in files that you download from Internet.
You're not gathering entropy, but just obfuscating the origin of your seed.

Ok, but at least we know that the complexity of the bits in the public photo are unique enough, so if we encrypt that, we know that same complexity in the encrypted bits should have the same complexity, but a different version of it.

Hard to explain, let me give you an analogy.


It would be like the analogy of using public dice, vs your own handcrafted dice:

• If you have your own handcrafted dice, that you crafted yourself from wood. Only you have it, and it's unique. It's not perfect, and it most likely has a bias, but the bias is unique to you, since you made it

VS

• Using a public dice that you borrow from a local casino, that has been used by at least 1000 people before you, and they had access to it, to measure it's bias, and deduct what kind of random numbers it gives out. But at least you know that the casino dice is manufactured high quality by default

So you have to choose between your own entropy, which will be low quality but private OR other people's entropy, which will be public, but high quality.

So you only need to encrypt that with a good secret key, and you remove the "public side of it" and you are left with the high quality entropy.



That is my thought process.

[BurtW]

All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source

This is a red flag for any security related algorithm or methodology.

What are you talking about it? We just keep the entropy secret, and all materials that we have used, will be deleted afterwards.

It must be kept secret otherwise it's not unique enough, and others can guess your passwords too.

Security though obscurity should only be used as a last resort.  If there is a way to gather entropy that does not rely on obscurity, and there are plenty, then you should use that method over any method that relies on obscurity.  This is security 101.

If your methodology relies on obscurity then it is implicitly less secure than a methodology that does not.

[piotr_n]

So you have to choose between your own entropy, which will be low quality but private OR other people's entropy, which will be public, but high quality.

Why would my own entropy be of a low quality?
It can have a perfect quality, if you just implement in properly.

Take a high resolution photo of objects that do not move, then take another one, without moving the camera...
Do you think the two photos will be identical?
Only for your eye - at the pixel level they will have plenty of differences.
And that's the noise - the natural form of entropy.

Even if you don't want to use the noise produced by A/D converters , you can still build an application that uses a noise created by nature.
* Take a photo of a cloudy sky, a running water, or a soil below your feet.
* Record any kind of noise with a microphone.
* Build your own simple hardware noise generator (ex)

The number of applications allowing to generate the actual entropy is only limited by your imagination.
And they would all be of quite a perfect quality, if you just don't screw it up.
Just take any of such input data, containing natural noise, put it through any cryptographic function producing high entropy output and you have quite a perfect random number generator. It's simple, cheap and very high quality. And it doesn't even need an internet connection.

[bob123]

You would gather a better entropy if you would use physical signals (like WLan rustle) or read randomly of the stack.
In Addition do that you should add some kind of random user input (Like movement of the mouse in the last X seconds, or seconds since Boot (But only if you start gathering your entropy manually and not directly after Boot via Autostart))

[Hydrogen]

Computers being state based deterministic machines, many cryptographers have used weather data and other information which appears to lack predictable patterns as a basis to generate random keys & make encryption more difficult to crack.

As far as I know, it is a common practice that has been in use for decades.

Good idea.

[shorena]

-snip-
That is my thought process.

You somehow assume asking a search engine will improve things, but it wont. You would just encode the information in a different way. Anyone that knows your keywords will find the same results[1]. If your initial keywords are not randomly generated, neither are your results. You also open yourself to several new attack vectors (e.g. MITM and Sybil) because you rely on information provided by others. Its significantly more difficult to manipulate a building you are about to take a picture off[2] than intercept your internet traffic and feed you predetermined data based on knowledge of your algorithm. If you really want to bake your own PRNG I suggest you follow piotr_n's advice. What you came up with might work for some time, until someone has a (strong) interest to make it work in their favor.

[1] within reason. There is a chance that different results will show up based on googles profiling or other factors.
[2] or the sound a busy intersection makes, etc.

[profitgenerator212]

-snip-
That is my thought process.

You somehow assume asking a search engine will improve things, but it wont. You would just encode the information in a different way. Anyone that knows your keywords will find the same results[1]. If your initial keywords are not randomly generated, neither are your results. You also open yourself to several new attack vectors (e.g. MITM and Sybil) because you rely on information provided by others. Its significantly more difficult to manipulate a building you are about to take a picture off[2] than intercept your internet traffic and feed you predetermined data based on knowledge of your algorithm. If you really want to bake your own PRNG I suggest you follow piotr_n's advice. What you came up with might work for some time, until someone has a (strong) interest to make it work in their favor.

[1] within reason. There is a chance that different results will show up based on googles profiling or other factors.
[2] or the sound a busy intersection makes, etc.

Ok I understand.

What about the encryption method? Is that a solid way to replace XOR, in the sense that you can't XOR thing together if the XOR key is shorter than the message.

So my intuition is to use encryption instead of XOR, that are not vulnerable to this attack?

So encrypting different inputs is a good cryptographic mixing method? Or should I use dedicated cryptographic mixing functions?

[gmaxwell]

-snip-
That is my thought process.

You somehow assume asking a search engine will improve things, but it wont. You would just encode the information in a different way. Anyone that knows your keywords will find the same results[1]. If your initial keywords are not randomly generated, neither are your results. You also open yourself to several new attack vectors (e.g. MITM and Sybil) because you rely on information provided by others. Its significantly more difficult to manipulate a building you are about to take a picture off[2] than intercept your internet traffic and feed you predetermined data based on knowledge of your algorithm. If you really want to bake your own PRNG I suggest you follow piotr_n's advice. What you came up with might work for some time, until someone has a (strong) interest to make it work in their favor.

[1] within reason. There is a chance that different results will show up based on googles profiling or other factors.
[2] or the sound a busy intersection makes, etc.

This can't be emphasized enough. The OP's fancy scheme is just a few more bits of key material (which they've now made public). Real attackers search not just over the words but over the methods, and this scheme has added little to no actual entropy-- it has just added complexity and serious additional leak risks.  It is the worst kind of security theater.

Unfortunately, I've found that when someone has gone down this rabbit hole they often become addicted to the complexity of their ritual-- like move "conspiracy wall"-- they weave together steps which don't help and sometimes hurt their security, but are unshakably convinced that it is the most secure method ever.  ... I've given up trying to convince them otherwise... but I comment so that someone who comes across this stuff with a spotless mind will not gain the impression that people think it is good it isn't.

[ImHash]

You should start developing an algorithm that if a message was viewed more than once some key words could change automatically you know like when you shine light on quantum particles they are waves but as long as you don't look at them they stay particles.

[BurtW]

Here are a few basic questions that come to mind:

Why do you think you need to re-invent this? 
Exactly what problem are you trying to solve here?
What do you think is the issue with all the other standard, vetted, secure methods for gathering entropy?

As everyone has told you in more ways than one, your system is overly complex and does not appear to do what you want, which is to securely gather entropy, correct?

[0xfff]

You'd be better off trusting random.org than doing some convoluted google search. If you have a physical user, just make them the source of randomness.

[piotr_n]

If you have a physical user, just make them the source of randomness.

you just need to mind that according to some people who are with us here, human brain is a very bad source of entropy, as there are (supposedly) multiple research papers proving that the output of one's brain can be easily guessed, simulated or predicted.

so, a physical user yes, but only if he isn't using his brain

in other words, if you tell him to hit random keys or move mouse around, it would be a good idea to find someone with parkinson's or other disease preventing his brain from controlling the muscles.

[BurtW]

I really liked your idea of using the camera on the system, taking some shots (or even a movie) of them waving their hands like a maniac, and gathering the entropy that way.

[piotr_n]

connecting a cheap mic and just recording the noise is probably much easier to implement.
and you can still clap your hands if you want.

I guess it's all about how many random bytes per second you need.
if many, go into video, otherwise audio should do just fine.