Bitcoin Forum

Bitcoin => Project Development => Topic started by: paladini on January 18, 2014, 02:41:13 AM



Title: Sorry Usebitcoins.info...
Post by: paladini on January 18, 2014, 02:41:13 AM
You got hacked...well, it's a joke - but you almost.

Today I'm releasing "Parse Usebitcoins.info (https://github.com/paladini/parse-usebitcoins)", a open-source project that aims to give free information to anyone that want it. With this little web crawler you can parse and get all links* from Usebitcoins, a popular Bitcoin directory. When we say links, we mean stores, services, goods and anything related to Bitcoin that have a link.


Before the gold, the basics:
Currently we have a lot of links in our database, and every link have the following attributes:

Quote
name # => The name of the store/service/item.
description # => Description of this store/service/item.
url # => The link itself.
image_url # => A link of the store/service/item image.
type # => Can be 'Services', 'Goods', 'Leisure' and 'Bitcoin'
category # => The category that this item belongs to (e.g: 'Electronics', 'Hosting services')

You can choose the better way to download this database: XML, JSON or SQL insertion script (MySQL and Potgresql supported). We don't know how much links we have, but is probably something above 1.700 links. Well, I think you already know the basics, let's advance. If you want something more, feel free to create a new issue (https://github.com/paladini/parse-usebitcoins/issues) with your tip / question.

At the moment, you have two ways to get this data: directly download and import this data to your database (MySQL insertion script, Postgresql insertion script, XML and JSON available) or download the tool and get the latest data.


Directly download the database:

If you want directly import the database to your website / software / service, you're on the rigth place.

  • MySQL Script - Temporarily down.
  • Postgresql Script - Temporarily down.
  • JSON (https://github.com/paladini/parse-usebitcoins/raw/master/public/parse_usebitcoins/data.json)
  • XML (https://github.com/paladini/parse-usebitcoins/raw/master/public/parse_usebitcoins/data.xml)

After download your desired database, just import it to your project. Don't know do that? MySQL Script (http://dev.mysql.com/doc/refman/5.0/en/mysql-batch-commands.html), Postgresql Script (http://stackoverflow.com/questions/9736085/run-a-postgresql-sql-file-using-command-line-args). XML and Json isn't language agnostic, so you must find a tutorial for your project language.

Download the tool and get latest data:

Check our guide here (https://github.com/paladini/parse-usebitcoins#directly-download-the-database).


Conclusion:

If you have any questions, let us know (https://github.com/paladini/parse-usebitcoins/issues/new). Hope people use this information to create great open-source projects :). If you create something with this data, please tell us. As there are more content related to Bitcoin, better.