Bitcoin Forum

Other => Meta => Topic started by: theymos on October 21, 2015, 07:40:32 AM



Title: Evil score visualization
Post by: theymos on October 21, 2015, 07:40:32 AM
When someone is banned, their IP and some of their neighboring IPs receive evil points. The thought occurred to me recently that you could create a map of the Internet according to evil points, and I couldn't resist doing this right away. The result is pretty cool-looking IMO. It also seems to show that the evil score system is working as expected: the vast majority of the Internet is not being forced to pay, and in the isolated sections where a registration fee is required, prohibitively-large fees are very rare.

Here's the image (zoom in):
https://bitcointalk.org/banmap201510.png

Each pixel is a /24 address block (ie. each pixel represents 256 IP addresses). The colors are:
█ Zero or nearly zero evil
█ A small amount of evil
█ More
█ More
█ At this point you actually have to pay if you register an account in this block
█ More
█ More
█ More
█ Pretty high
█ A ton of evil, more than anyone is likely to pay

This is per block, so a single IP address could have an evil score requiring payment while its block still shows up as black here. A colored pixel indicates the evil score of a typical IP in that block.

Addresses are laid out in the standard way. So you can for example cross-reference with these maps: https://ant.isi.edu/address/

A /24 should almost never uniquely identify someone, but to be safe I randomly added, removed, and modified some of this data for plausible deniability.


Title: Re: Evil score visualization
Post by: botany on October 21, 2015, 07:50:11 AM
When someone is banned, their IP and some of their neighboring IPs receive evil points.

Just curious - Is it the last used IP? What happens when the IP an account registered with, is different from the IPs used later on?


Title: Re: Evil score visualization
Post by: ndnh on October 21, 2015, 07:57:11 AM
When someone is banned, their IP and some of their neighboring IPs receive evil points.

Just curious - Is it the last used IP? What happens when the IP an account registered with, is different from the IPs used later on?

It is most probably the IP address the account is registered with.


Title: Re: Evil score visualization
Post by: Swordsoffreedom on October 21, 2015, 07:57:49 AM
Do evil points ever fade away over time or is the map consistent for certain ranges.
And it looks like the Internet is not so evil after all except for that one well defined corner  ;D


Title: Re: Evil score visualization
Post by: dogie on October 21, 2015, 08:00:30 AM
Do evil points ever fade away or is the map consistent forever.
And it looks like the Internet is not so evil after all except for that one well defined corner  ;D
Last I heard they do decay.


@Theymos is there a way to convert/anon the data such that it could be released? I've never visualised something of only nodes, would be interesting to see what Gephi can do.


Title: Re: Evil score visualization
Post by: theymos on October 21, 2015, 08:00:57 AM
Just curious - Is it the last used IP?

All IPs on file for them are banned, and each one will affect its neighbors.

Do evil points ever fade away

Yes. This map is pretty short-term. And I'm probably not going to update it. But it's still interesting IMO.


Title: Re: Evil score visualization
Post by: theymos on October 21, 2015, 08:06:40 AM
@Theymos is there a way to convert/anon the data such that it could be released?

You can already reconstruct a lot of the data from the graph with some work. Given a pixel position, you can calculate its /24 address. I'm hesitant to publish the source data because, given that, it's more likely that you'll be able to figure out which data entries I fudged.


Title: Re: Evil score visualization
Post by: Lauda on October 21, 2015, 08:30:13 AM
Looks pretty good. How long did it take you to visualize it? I always wonder why you never tend to update (points @reported posts and accuracy). Honestly, I was expecting more red blocks.


Title: Re: Evil score visualization
Post by: dogie on October 21, 2015, 10:34:34 AM
@Theymos is there a way to convert/anon the data such that it could be released?

You can already reconstruct a lot of the data from the graph with some work. Given a pixel position, you can calculate its /24 address. I'm hesitant to publish the source data because, given that, it's more likely that you'll be able to figure out which data entries I fudged.

I'm not understanding why it is an issue if someone could work out evil IP ranges. The high ones are almost all going to be VPNs and it doesn't identify anyone. The years of data may be useful to others operating other bitcoin services to pre-empt problems before they occur. Regardless, deconstructing the image is beyond my paygrade so unless someone else can get me the pixel data into a written data form I'll have to pass.


Title: Re: Evil score visualization
Post by: GermanGiant on October 21, 2015, 01:40:18 PM
Looks like a Nebula.

Here's the image (zoom in):
https://bitcointalk.org/banmap201510.png

What does X & Y axis stand for in this image ?


Title: Re: Evil score visualization
Post by: whywefight on October 21, 2015, 01:48:49 PM
any chance to get this on a world map?


Title: Re: Evil score visualization
Post by: --Encrypted-- on October 21, 2015, 02:45:35 PM
having a hard time understanding this.
so the top left would be xx.0.0.0 and xx.255.255.255 for the top right pixel, and xx.170.130.179 for the blue dot near the bottom right... am I correct?


Title: Re: Evil score visualization
Post by: theymos on October 21, 2015, 04:04:56 PM
having a hard time understanding this.
so the top left would be xx.0.0.0 and xx.255.255.255 for the top right pixel, and xx.170.130.179 for the blue dot near the bottom right... am I correct?

The numbers are right, but it's 0.0.0.x, 255.255.255.x, and 170.130.179.x.


Title: Re: Evil score visualization
Post by: --Encrypted-- on October 21, 2015, 04:30:42 PM
having a hard time understanding this.
so the top left would be xx.0.0.0 and xx.255.255.255 for the top right pixel, and xx.170.130.179 for the blue dot near the bottom right... am I correct?

The numbers are right, but it's 0.0.0.x, 255.255.255.x, and 170.130.179.x.

got it. thanks.

@dogie
splitting the image to 256 16x16 pics can make things a lot easier. though I'm not sure how to do it afterwards in a simple way.


Title: Re: Evil score visualization
Post by: dothebeats on October 21, 2015, 05:02:32 PM
At first I thought I was looking at a massive void in space lol.

any chance to get this on a world map?

A great idea, though I wonder how it can be done with such a massive number of IP addresses available. When I first registered an account, I have had to pay a mere 8752 satoshi. I guess my IP back then contains some units of evil more than the blue green threshold.


Title: Re: Evil score visualization
Post by: jacee on October 21, 2015, 05:19:15 PM
any chance to get this on a world map?

This will be a lot more easier to understand if it will be on the world map. The pixels looks like a bunch of shining stars to me. LOL. Great idea tho, I once encountered to pay a fee when I used a vpn before. So this is why..


Title: Re: Evil score visualization
Post by: unamis76 on October 21, 2015, 09:22:31 PM
Nice map, pretty interesting data :) A must have for forum users that are number and statistics freaks, I guess :D


Title: Re: Evil score visualization
Post by: dogie on October 22, 2015, 12:21:06 AM
having a hard time understanding this.
so the top left would be xx.0.0.0 and xx.255.255.255 for the top right pixel, and xx.170.130.179 for the blue dot near the bottom right... am I correct?

The numbers are right, but it's 0.0.0.x, 255.255.255.x, and 170.130.179.x.

got it. thanks.

@dogie
splitting the image to 256 16x16 pics can make things a lot easier. though I'm not sure how to do it afterwards in a simple way.

Doesn't particularly help me as I don't have any way to automate the process after that. I have to start with words and turn words into other things as nothing I know interacts with non words.


Title: Re: Evil score visualization
Post by: MrPump on October 22, 2015, 12:26:17 AM
So if you get permbanned you have to pay ? Or can you ie pay to have ban killed sooner than given date?


Title: Re: Evil score visualization
Post by: dogie on October 22, 2015, 12:55:12 AM
So if you get permbanned you have to pay ? Or can you ie pay to have ban killed sooner than given date?
If an IP has someone banned on it then for someone else to register via the same IP they have to pay a bit. Its an alternative to outright IP bans.


Title: Re: Evil score visualization
Post by: MrPump on October 22, 2015, 02:08:46 AM
So if you get permbanned you have to pay ? Or can you ie pay to have ban killed sooner than given date?
If an IP has someone banned on it then for someone else to register via the same IP they have to pay a bit. Its an alternative to outright IP bans.

And what if the account is banned for lets say 30-60 days [or it doesnt say].
What happends there?

1 of my members wrote me hes banned but he doesnt know for how long


Title: Re: Evil score visualization
Post by: --Encrypted-- on October 22, 2015, 02:30:02 AM
And what if the account is banned for lets say 30-60 days [or it doesnt say].
What happends there?


the IP of the banned account will get some evil units and you'll have to wait until the account gets unbanned to post. so to answer your previous question, you can't get unbanned by paying. you only have to pay to enable posting with a new account that was registered with an IP that was blacklisted by this forum.


Title: Re: Evil score visualization
Post by: dothebeats on October 22, 2015, 03:40:10 AM
And what if the account is banned for lets say 30-60 days [or it doesnt say].
What happends there?


the IP of the banned account will get some evil units and you'll have to wait until the account gets unbanned to post. so to answer your previous question, you can't get unbanned by paying. you only have to pay to enable posting with a new account that was registered with an IP that was blacklisted by this forum.

I think it is not IP-related anymore when it comes to bans but rather the account itself is banned. A ban without duration could be a permaban and CAN only be removed by posting an appeal here on Meta about the ban and the mods will decide whether the ban should be lifted or not.


Title: Re: Evil score visualization
Post by: rebuilder on October 23, 2015, 09:14:41 PM
Doesn't particularly help me as I don't have any way to automate the process after that. I have to start with words and turn words into other things as nothing I know interacts with non words.

dogie, for a rough solution you could use ImageMagick's convert to read the pixel value for each pixel:
http://apple.stackexchange.com/questions/26719/any-tool-to-tell-me-the-exact-color-value-at-a-point-on-a-png-image

the relevant part:

Quote
$ convert image.png -crop '1x1+100+200' txt:-

Outputs, e.g.

# ImageMagick pixel enumeration: 1,1,255,rgb
0,0: (236,236,236)  #ECECEC  rgb(236,236,236)

then you need to correlate the output with IP based on the mapping specified here before.

This is a pretty ugly way to do it, but it should require minimal coding, so for a one-off, it's how I'd do it.


Title: Re: Evil score visualization
Post by: dogie on October 25, 2015, 06:22:48 AM
Doesn't particularly help me as I don't have any way to automate the process after that. I have to start with words and turn words into other things as nothing I know interacts with non words.

dogie, for a rough solution you could use ImageMagick's convert to read the pixel value for each pixel:
http://apple.stackexchange.com/questions/26719/any-tool-to-tell-me-the-exact-color-value-at-a-point-on-a-png-image

the relevant part:

Quote
$ convert image.png -crop '1x1+100+200' txt:-

Outputs, e.g.

# ImageMagick pixel enumeration: 1,1,255,rgb
0,0: (236,236,236)  #ECECEC  rgb(236,236,236)

then you need to correlate the output with IP based on the mapping specified here before.

This is a pretty ugly way to do it, but it should require minimal coding, so for a one-off, it's how I'd do it.

Nice, worked perfectly. If I get bored I'll see what I can do with this although at this point I'm not actually sure :P


Title: Re: Evil score visualization
Post by: Gleb Gamow on October 25, 2015, 06:53:21 AM
Quick aside: NERDS!  ;D


Title: Re: Evil score visualization
Post by: dogie on November 30, 2015, 04:07:40 PM
For reference, I got pretty close to outputting results but couldn't find a way to reverse the 12th order Hilbert curve in excel. There is some code to do it here (https://en.wikipedia.org/wiki/Hilbert_curve#Applications_and_mapping_algorithms) but I'm unsure how to get that usable.


Title: Re: Evil score visualization
Post by: grue on December 06, 2015, 10:11:32 PM
If anyone's interested I made a program that mapped each pixel to an ASN, and counted how many evils each ASN has. It's available as a CSV file (https://gist.github.com/anonymous/3c8df2f906fd58a5d786). The first column is the ASN, the other columns are the number of pixels for that particular color. The (n+1)th column correspond to nth color listed by theymos. (ie. the second column corresponds to the number of black pixels and the 8th column corresponds to the number of pink pixels for a particular ASN).

edit:
forgot to mention that in some cases, a pixel (/24 subnet) correspond to multiple ASNs, or that an ASN is only responsible for part of a /24. In those cases, the color count is incremented proportionally to the size of the subnet they control. So if a ASN controls 10.0.0.0/26, and the corresponding pixel color is white, then the 3rd column (the column for white) for that ASN will be incremented by 0.25, since a /26 subnet is quarter of the size of a /24.


Title: Re: Evil score visualization
Post by: Luxurycoin_ICO on November 20, 2017, 10:15:54 AM
how to remove it ? cause fee is very small, hard to make a transaction


Title: Re: Evil score visualization
Post by: Coin.Master636 on November 20, 2017, 10:30:36 AM
This is truly awesome. I really love it!! But I don't like IP banning... IP's are just not personal enough... Especially with dynamic IP's with internet providers...
I like things as Login Through Google to identify unique persons. But obviously there can be more than one Google account for each person...