Darpa Is Developing a Search Engine for the Dark Web

[sgk]

How big is that?

http://www.wired.com/2015/02/darpa-memex-dark-web/

A new search engine being developed by Darpa aims to shine a light on the dark web and uncover patterns and relationships in online data to help law enforcement and others track illegal activity.

At present, the dark web is *assumed* to be 4000 times bigger than regular web that can be accessed without TOR.
However this is only a guestimate because without proper search tools it is virtually impossible to make any sensible estimate of the size of DarkWeb.

When this starts to work, we can have a better idea of how big it is, and most importantly how much illegal activities actually go in there.

[b!z]

How big is that?

It's very small. Where did you get 4000x from? I really doubt that is true.

It probably refers to the number of ALL sites not accessible on the public web for whatever reason like company intranets etc., not ONLY Tor.

[sgk]

How big is that?

It's very small. Where did you get 4000x from? I really doubt that is true.

It probably refers to the number of ALL sites not accessible on the public web for whatever reason like company intranets etc., not ONLY Tor.

I remember reading about it on Wikipedia some time back. I just went through it again to verify.
I was mistaken in sighting 4000, it is actually 400 to 550 according to Wikipedia. Still, that's too large.

Early estimates suggested that the deep web is 400 to 550 times larger than the surface web. However, since more information and sites are always being added, it can be assumed that the deep web is growing exponentially at a rate that cannot be quantified.

Quote from:
http://en.wikipedia.org/wiki/Deep_Web

Oh look it's this common misunderstanding again.

When people refer to the "dark web" being bigger than the internet, they are not referring to Tor hidden services and the "dark" or "deep" web often referred to by the media.[...]

Thanks, I was also not pretty clear about this concept. You learn something new everyday

[RealMalatesta]

It is so funny - I was at a conference, and a cop showed a slide, showing the Darkweb with all the big bad players being 80% of the Internet. "So if it is 'dark', how comes you know how big it is?" Silence.
Fact is that also databases which can only be accessed by paying customers are part of the "Darkweb" because they are not indexed by search engines.  So the term "DarkWeb" is mainly a PR-stunt by some (political/legal) control freaks, nothing more.

[pedrog]

How big is that?

It's very small. Where did you get 4000x from? I really doubt that is true.

It probably refers to the number of ALL sites not accessible on the public web for whatever reason like company intranets etc., not ONLY Tor.

I think he meant deep web...

[RealMalatesta]

How big is that?

It's very small. Where did you get 4000x from? I really doubt that is true.

It probably refers to the number of ALL sites not accessible on the public web for whatever reason like company intranets etc., not ONLY Tor.

I think he meant deep web...

Well, Memex, as defined by Darpa, should index the deep web. And this includes the dark web. However, Darpa wouldn't be the first to fail...

[sgk]

Some new figures on size:

"The newly launched Tor search engine called Onion.city—probably the finest Tor search engine in existence—is indexing about 350,000 pages, and it's only been around for less than a week."

http://www.dailydot.com/politics/tor-dark-net-study-size/

[vm1990]

considering the darknet is used for both illegal and legal things not to mention activists and many many LEA agencies this is probably a stupid idea but wikileaks will have a much easier job and so will hackers. stupid people only think of stupid ideas.