Bitcoin Forum
March 28, 2024, 10:28:03 PM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 »  All
  Print  
Author Topic: Is my method a secure way to gather entropy?  (Read 2113 times)
profitgenerator212 (OP)
Sr. Member
****
Offline Offline

Activity: 714
Merit: 251



View Profile
February 06, 2017, 05:29:14 AM
 #1

I have invented a new way of gathering entropy, in order to generate keys from it, I it all described here in detail:

https://steemit.com/security/@profitgenerator/generate-secure-random-passwords-or-private-keys

In summary the method is this:

  • Use the internet to find public data on it that has large information content: files,music, videos, etc.. , or add your own if possible
  • All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source
  • We gather X number files from a search engine using random keywords, we gather another P number files , where X = P
  • Instead of using XOR, due to fears of short key, we just encrypt all the P files with AES256CBC with separate keys generated with the OS's default RNG, the AES256CBC encrypted files will be E
  • Then we take the 512 bit hash of the E files, add a random salt to each of them, and use these as separate AES256CBC encryption keys with which we encrypt all the X files separately with AES256CBC
  • Then all encrypted X file are already a unique good source of randomness, then we put all X files in a .tar file, and this is our MAINENTROPY.tar file

So when we just want to create a password, we just take the SHA1,SHA256, or SHA512 bit hash of the MAINENTROPY.tar file and that will be our private key.

After use, we just add another random file inside the archive, to act as a salt or nonce, and produce a new unique hash next time we use it.



What do you think?


                            █████
                        █████████████
                     █████████████
                 ██████████████        █████
              █████████████        ████████████
          ██████████████        █████████████
       █████████████        █████████████       ██████
       ██████████        ████████████           ██████
       ███████       █████████████       ███    ██████
       ███████    █████████████       ██████    ██████
       ████████████████████       ██████████    ██████
       █████████████████       █████████████    ██████
       █████████████       █████████████        ██████
       ██████████       █████████████           ██████
       ███████      ██████████████       ███    ██████
       ██████    █████████████       ███████    ██████
       ██████    ██████████       ██████████    ██████
       ██████    ██████        █████████████    ██████
       ██████    ███       █████████████        ██████
       ██████           █████████████       ██████████
       ██████       █████████████        █████████████
                 █████████████       █████████████
              ████████████        █████████████
                  ████         ████████████
                           █████████████
                         ███████████
                            █████
Ferrum Network • Interoperability Network for Financial Applications
Make sure you back up your wallet regularly! Unlike a bank account, nobody can help you if you lose access to your BTC.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1711664883
Hero Member
*
Offline Offline

Posts: 1711664883

View Profile Personal Message (Offline)

Ignore
1711664883
Reply with quote  #2

1711664883
Report to moderator
1711664883
Hero Member
*
Offline Offline

Posts: 1711664883

View Profile Personal Message (Offline)

Ignore
1711664883
Reply with quote  #2

1711664883
Report to moderator
1711664883
Hero Member
*
Offline Offline

Posts: 1711664883

View Profile Personal Message (Offline)

Ignore
1711664883
Reply with quote  #2

1711664883
Report to moderator
cr1776
Legendary
*
Offline Offline

Activity: 3990
Merit: 1295


View Profile
February 06, 2017, 05:11:16 PM
Last edit: February 07, 2017, 12:44:22 AM by cr1776
Merited by ABCbits (1)
 #2

"Use the internet to find public data" - using TOR?  New sessions each time?  (to avoid someone monitoring your searches or your ISP monitoring your downloads)?   Security through obscurity is worrisome.

Seems complicated for the benefits.  Cheesy


I have invented a new way of gathering entropy, in order to generate keys from it, I it all described here in detail:

https://steemit.com/security/@profitgenerator/generate-secure-random-passwords-or-private-keys

In summary the method is this:

  • Use the internet to find public data on it that has large information content: files,music, videos, etc.. , or add your own if possible
  • All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source
  • We gather X number files from a search engine using random keywords, we gather another P number files , where X = P
  • Instead of using XOR, due to fears of short key, we just encrypt all the P files with AES256CBC with separate keys generated with the OS's default RNG, the AES256CBC encrypted files will be E
  • Then we take the 512 bit hash of the E files, add a random salt to each of them, and use these as separate AES256CBC encryption keys with which we encrypt all the X files separately with AES256CBC
  • Then all encrypted X file are already a unique good source of randomness, then we put all X files in a .tar file, and this is our MAINENTROPY.tar file

So when we just want to create a password, we just take the SHA1,SHA256, or SHA512 bit hash of the MAINENTROPY.tar file and that will be our private key.

After use, we just add another random file inside the archive, to act as a salt or nonce, and produce a new unique hash next time we use it.



What do you think?
piotr_n
Legendary
*
Offline Offline

Activity: 2053
Merit: 1344


aka tonikt


View Profile WWW
February 06, 2017, 05:51:59 PM
Merited by ABCbits (1)
 #3

you're overcomplicaing it, imho

you'd be better off just taking photos or recording voice, instead of downloading existing media files from internet.

each analog-to-digital converter adds some noise to the output data.
with a proper application, it can actually be used as a pretty perfect random number source.

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
BurtW
Legendary
*
Offline Offline

Activity: 2646
Merit: 1129

All paid signature campaigns should be banned.


View Profile WWW
February 07, 2017, 12:12:46 AM
 #4

All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source

This is a red flag for any security related algorithm or methodology.

Our family was terrorized by Homeland Security.  Read all about it here:  http://www.jmwagner.com/ and http://www.burtw.com/  Any donations to help us recover from the $300,000 in legal fees and forced donations to the Federal Asset Forfeiture slush fund are greatly appreciated!
profitgenerator212 (OP)
Sr. Member
****
Offline Offline

Activity: 714
Merit: 251



View Profile
February 10, 2017, 02:41:52 AM
 #5

"Use the internet to find public data" - using TOR?  New sessions each time?  (to avoid someone monitoring your searches or your ISP monitoring your downloads)?   Security through obscurity is worrisome.

Seems complicated for the benefits.  Cheesy


Not necessarly, because the data is already out there, so public data is already public data.Just like if you are english speaking, and your mnemonic phrase consists of english words, the attacker won't try to brute force chinese words.

The same way even though by downloading the files over a monitored internet connection, a targeted attacker could know what you are up to, but still can't guess the final entropy.

Or in other words, the haystack gets smaller, but it's still big enough to provide good security in my opinion.

you're overcomplicaing it, imho

you'd be better off just taking photos or recording voice, instead of downloading existing media files from internet.

each analog-to-digital converter adds some noise to the output data.
with a proper application, it can actually be used as a pretty perfect random number source.

That is almost the same as downloading from the internet.

If you go out in your city, and start photographing buildings, that is exactly the same thing as downloading over the internet, minus the fact that a targeted attacker doesn't know what you will do with the photos, but the photos themselves are already public, because anyone else can take them (with smaller or larger variations of angle, brightness, etc...)

But a phrase might not be unique enough, so we need to combine multiple sources of uniqueness.

All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source

This is a red flag for any security related algorithm or methodology.

What are you talking about it? We just keep the entropy secret, and all materials that we have used, will be deleted afterwards.

It must be kept secret otherwise it's not unique enough, and others can guess your passwords too.


                            █████
                        █████████████
                     █████████████
                 ██████████████        █████
              █████████████        ████████████
          ██████████████        █████████████
       █████████████        █████████████       ██████
       ██████████        ████████████           ██████
       ███████       █████████████       ███    ██████
       ███████    █████████████       ██████    ██████
       ████████████████████       ██████████    ██████
       █████████████████       █████████████    ██████
       █████████████       █████████████        ██████
       ██████████       █████████████           ██████
       ███████      ██████████████       ███    ██████
       ██████    █████████████       ███████    ██████
       ██████    ██████████       ██████████    ██████
       ██████    ██████        █████████████    ██████
       ██████    ███       █████████████        ██████
       ██████           █████████████       ██████████
       ██████       █████████████        █████████████
                 █████████████       █████████████
              ████████████        █████████████
                  ████         ████████████
                           █████████████
                         ███████████
                            █████
Ferrum Network • Interoperability Network for Financial Applications
piotr_n
Legendary
*
Offline Offline

Activity: 2053
Merit: 1344


aka tonikt


View Profile WWW
February 10, 2017, 03:30:05 AM
 #6

No it isn't the same thing.

When you make a photo yourself, nobody else has its copy.

You asked about 'gathering entropy'.
There is no entropy in files that you download from Internet.
You're not gathering entropy, but just obfuscating the origin of your seed.

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
profitgenerator212 (OP)
Sr. Member
****
Offline Offline

Activity: 714
Merit: 251



View Profile
February 10, 2017, 10:01:00 AM
 #7

No it isn't the same thing.

When you make a photo yourself, nobody else has its copy.

You asked about 'gathering entropy'.
There is no entropy in files that you download from Internet.
You're not gathering entropy, but just obfuscating the origin of your seed.


Ok, but at least we know that the complexity of the bits in the public photo are unique enough, so if we encrypt that, we know that same complexity in the encrypted bits should have the same complexity, but a different version of it.

Hard to explain, let me give you an analogy.


It would be like the analogy of using public dice, vs your own handcrafted dice:
  • If you have your own handcrafted dice, that you crafted yourself from wood. Only you have it, and it's unique. It's not perfect, and it most likely has a bias, but the bias is unique to you, since you made it
VS
  • Using a public dice that you borrow from a local casino, that has been used by at least 1000 people before you, and they had access to it, to measure it's bias, and deduct what kind of random numbers it gives out. But at least you know that the casino dice is manufactured high quality by default




So you have to choose between your own entropy, which will be low quality but private OR other people's entropy, which will be public, but high quality.

So you only need to encrypt that with a good secret key, and you remove the "public side of it" and you are left with the high quality entropy.



That is my thought process.


                            █████
                        █████████████
                     █████████████
                 ██████████████        █████
              █████████████        ████████████
          ██████████████        █████████████
       █████████████        █████████████       ██████
       ██████████        ████████████           ██████
       ███████       █████████████       ███    ██████
       ███████    █████████████       ██████    ██████
       ████████████████████       ██████████    ██████
       █████████████████       █████████████    ██████
       █████████████       █████████████        ██████
       ██████████       █████████████           ██████
       ███████      ██████████████       ███    ██████
       ██████    █████████████       ███████    ██████
       ██████    ██████████       ██████████    ██████
       ██████    ██████        █████████████    ██████
       ██████    ███       █████████████        ██████
       ██████           █████████████       ██████████
       ██████       █████████████        █████████████
                 █████████████       █████████████
              ████████████        █████████████
                  ████         ████████████
                           █████████████
                         ███████████
                            █████
Ferrum Network • Interoperability Network for Financial Applications
BurtW
Legendary
*
Offline Offline

Activity: 2646
Merit: 1129

All paid signature campaigns should be banned.


View Profile WWW
February 10, 2017, 02:31:03 PM
Merited by ABCbits (1)
 #8


All files kept as secret as possible, for obvious reasons, if we want this to be our entropy source

This is a red flag for any security related algorithm or methodology.

What are you talking about it? We just keep the entropy secret, and all materials that we have used, will be deleted afterwards.

It must be kept secret otherwise it's not unique enough, and others can guess your passwords too.
Security though obscurity should only be used as a last resort.  If there is a way to gather entropy that does not rely on obscurity, and there are plenty, then you should use that method over any method that relies on obscurity.  This is security 101.

If your methodology relies on obscurity then it is implicitly less secure than a methodology that does not.

Our family was terrorized by Homeland Security.  Read all about it here:  http://www.jmwagner.com/ and http://www.burtw.com/  Any donations to help us recover from the $300,000 in legal fees and forced donations to the Federal Asset Forfeiture slush fund are greatly appreciated!
piotr_n
Legendary
*
Offline Offline

Activity: 2053
Merit: 1344


aka tonikt


View Profile WWW
February 10, 2017, 02:43:00 PM
Last edit: February 10, 2017, 03:11:57 PM by piotr_n
Merited by ABCbits (3)
 #9

So you have to choose between your own entropy, which will be low quality but private OR other people's entropy, which will be public, but high quality.

Why would my own entropy be of a low quality?
It can have a perfect quality, if you just implement in properly.

Take a high resolution photo of objects that do not move, then take another one, without moving the camera...
Do you think the two photos will be identical?
Only for your eye - at the pixel level they will have plenty of differences.
And that's the noise - the natural form of entropy.

Even if you don't want to use the noise produced by A/D converters , you can still build an application that uses a noise created by nature.
* Take a photo of a cloudy sky, a running water, or a soil below your feet.
* Record any kind of noise with a microphone.
* Build your own simple hardware noise generator (ex)

The number of applications allowing to generate the actual entropy is only limited by your imagination.
And they would all be of quite a perfect quality, if you just don't screw it up.
Just take any of such input data, containing natural noise, put it through any cryptographic function producing high entropy output and you have quite a perfect random number generator. It's simple, cheap and very high quality. And it doesn't even need an internet connection.

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
bob123
Legendary
*
Offline Offline

Activity: 1624
Merit: 2481



View Profile WWW
February 11, 2017, 05:33:14 PM
 #10

You would gather a better entropy if you would use physical signals (like WLan rustle) or read randomly of the stack.
In Addition do that you should add some kind of random user input (Like movement of the mouse in the last X seconds, or seconds since Boot (But only if you start gathering your entropy manually and not directly after Boot via Autostart))

Hydrogen
Legendary
*
Offline Offline

Activity: 2562
Merit: 1441



View Profile
February 13, 2017, 08:03:09 AM
 #11

Computers being state based deterministic machines, many cryptographers have used weather data and other information which appears to lack predictable patterns as a basis to generate random keys & make encryption more difficult to crack.

As far as I know, it is a common practice that has been in use for decades.

Good idea.
shorena
Copper Member
Legendary
*
Offline Offline

Activity: 1498
Merit: 1480


No I dont escrow anymore.


View Profile WWW
February 13, 2017, 08:23:11 AM
 #12

-snip-
That is my thought process.

You somehow assume asking a search engine will improve things, but it wont. You would just encode the information in a different way. Anyone that knows your keywords will find the same results[1]. If your initial keywords are not randomly generated, neither are your results. You also open yourself to several new attack vectors (e.g. MITM and Sybil) because you rely on information provided by others. Its significantly more difficult to manipulate a building you are about to take a picture off[2] than intercept your internet traffic and feed you predetermined data based on knowledge of your algorithm. If you really want to bake your own PRNG I suggest you follow piotr_n's advice. What you came up with might work for some time, until someone has a (strong) interest to make it work in their favor.

[1] within reason. There is a chance that different results will show up based on googles profiling or other factors.
[2] or the sound a busy intersection makes, etc.

Im not really here, its just your imagination.
profitgenerator212 (OP)
Sr. Member
****
Offline Offline

Activity: 714
Merit: 251



View Profile
February 14, 2017, 11:46:39 AM
 #13

-snip-
That is my thought process.

You somehow assume asking a search engine will improve things, but it wont. You would just encode the information in a different way. Anyone that knows your keywords will find the same results[1]. If your initial keywords are not randomly generated, neither are your results. You also open yourself to several new attack vectors (e.g. MITM and Sybil) because you rely on information provided by others. Its significantly more difficult to manipulate a building you are about to take a picture off[2] than intercept your internet traffic and feed you predetermined data based on knowledge of your algorithm. If you really want to bake your own PRNG I suggest you follow piotr_n's advice. What you came up with might work for some time, until someone has a (strong) interest to make it work in their favor.

[1] within reason. There is a chance that different results will show up based on googles profiling or other factors.
[2] or the sound a busy intersection makes, etc.

Ok I understand.

What about the encryption method? Is that a solid way to replace XOR, in the sense that you can't XOR thing together if the XOR key is shorter than the message.

So my intuition is to use encryption instead of XOR, that are not vulnerable to this attack?

So encrypting different inputs is a good cryptographic mixing method? Or should I use dedicated cryptographic mixing functions?


                            █████
                        █████████████
                     █████████████
                 ██████████████        █████
              █████████████        ████████████
          ██████████████        █████████████
       █████████████        █████████████       ██████
       ██████████        ████████████           ██████
       ███████       █████████████       ███    ██████
       ███████    █████████████       ██████    ██████
       ████████████████████       ██████████    ██████
       █████████████████       █████████████    ██████
       █████████████       █████████████        ██████
       ██████████       █████████████           ██████
       ███████      ██████████████       ███    ██████
       ██████    █████████████       ███████    ██████
       ██████    ██████████       ██████████    ██████
       ██████    ██████        █████████████    ██████
       ██████    ███       █████████████        ██████
       ██████           █████████████       ██████████
       ██████       █████████████        █████████████
                 █████████████       █████████████
              ████████████        █████████████
                  ████         ████████████
                           █████████████
                         ███████████
                            █████
Ferrum Network • Interoperability Network for Financial Applications
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4158
Merit: 8343



View Profile WWW
February 15, 2017, 09:51:19 AM
Merited by ABCbits (2)
 #14

-snip-
That is my thought process.

You somehow assume asking a search engine will improve things, but it wont. You would just encode the information in a different way. Anyone that knows your keywords will find the same results[1]. If your initial keywords are not randomly generated, neither are your results. You also open yourself to several new attack vectors (e.g. MITM and Sybil) because you rely on information provided by others. Its significantly more difficult to manipulate a building you are about to take a picture off[2] than intercept your internet traffic and feed you predetermined data based on knowledge of your algorithm. If you really want to bake your own PRNG I suggest you follow piotr_n's advice. What you came up with might work for some time, until someone has a (strong) interest to make it work in their favor.

[1] within reason. There is a chance that different results will show up based on googles profiling or other factors.
[2] or the sound a busy intersection makes, etc.


This can't be emphasized enough. The OP's fancy scheme is just a few more bits of key material (which they've now made public). Real attackers search not just over the words but over the methods, and this scheme has added little to no actual entropy-- it has just added complexity and serious additional leak risks.  It is the worst kind of security theater.

Unfortunately, I've found that when someone has gone down this rabbit hole they often become addicted to the complexity of their ritual-- like move "conspiracy wall"-- they weave together steps which don't help and sometimes hurt their security, but are unshakably convinced that it is the most secure method ever.  ... I've given up trying to convince them otherwise... but I comment so that someone who comes across this stuff with a spotless mind will not gain the impression that people think it is good it isn't.

ImHash
Hero Member
*****
Offline Offline

Activity: 924
Merit: 506


View Profile
February 15, 2017, 03:27:22 PM
 #15

You should start developing an algorithm that if a message was viewed more than once some key words could change automatically you know like when you shine light on quantum particles they are waves but as long as you don't look at them they stay particles.
BurtW
Legendary
*
Offline Offline

Activity: 2646
Merit: 1129

All paid signature campaigns should be banned.


View Profile WWW
February 15, 2017, 06:37:39 PM
Merited by ABCbits (1)
 #16

Here are a few basic questions that come to mind:

Why do you think you need to re-invent this? 
Exactly what problem are you trying to solve here?
What do you think is the issue with all the other standard, vetted, secure methods for gathering entropy?

As everyone has told you in more ways than one, your system is overly complex and does not appear to do what you want, which is to securely gather entropy, correct?

Our family was terrorized by Homeland Security.  Read all about it here:  http://www.jmwagner.com/ and http://www.burtw.com/  Any donations to help us recover from the $300,000 in legal fees and forced donations to the Federal Asset Forfeiture slush fund are greatly appreciated!
0xfff
Full Member
***
Offline Offline

Activity: 224
Merit: 100


View Profile
February 15, 2017, 07:01:24 PM
 #17

You'd be better off trusting random.org than doing some convoluted google search. If you have a physical user, just make them the source of randomness.
piotr_n
Legendary
*
Offline Offline

Activity: 2053
Merit: 1344


aka tonikt


View Profile WWW
February 15, 2017, 08:23:15 PM
Last edit: February 15, 2017, 08:44:54 PM by piotr_n
 #18

If you have a physical user, just make them the source of randomness.
you just need to mind that according to some people who are with us here, human brain is a very bad source of entropy, as there are (supposedly) multiple research papers proving that the output of one's brain can be easily guessed, simulated or predicted.

so, a physical user yes, but only if he isn't using his brain Wink

in other words, if you tell him to hit random keys or move mouse around, it would be a good idea to find someone with parkinson's or other disease preventing his brain from controlling the muscles. Smiley

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
BurtW
Legendary
*
Offline Offline

Activity: 2646
Merit: 1129

All paid signature campaigns should be banned.


View Profile WWW
February 16, 2017, 02:27:54 AM
 #19

I really liked your idea of using the camera on the system, taking some shots (or even a movie) of them waving their hands like a maniac, and gathering the entropy that way.

Our family was terrorized by Homeland Security.  Read all about it here:  http://www.jmwagner.com/ and http://www.burtw.com/  Any donations to help us recover from the $300,000 in legal fees and forced donations to the Federal Asset Forfeiture slush fund are greatly appreciated!
piotr_n
Legendary
*
Offline Offline

Activity: 2053
Merit: 1344


aka tonikt


View Profile WWW
February 16, 2017, 02:37:04 AM
 #20

connecting a cheap mic and just recording the noise is probably much easier to implement.
and you can still clap your hands if you want. Smiley

I guess it's all about how many random bytes per second you need.
if many, go into video, otherwise audio should do just fine.

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!