Bitcoin Forum

Bitcoin => Project Development => Topic started by: cryptohappening on November 10, 2019, 12:01:18 PM



Title: Crypto News Aggregator
Post by: cryptohappening on November 10, 2019, 12:01:18 PM
I posted a web application that I created a while back that received some positive feedback. I have since updated to include some of the suggestions. Features now include:

  • Cryptocurrency and stock market news aggregator
  • Filter for the aggregated news so you can filter between cryptocurrency and stock market news sources. News sources from reddit can also be filtered.
  • Cryptocurrency and stock market overview
  • Cryptocurrency market table
  • Growth history and all time high data
  • Historical charts with trend lines
  • Bitcoin live ticker
  • Bitcoin sentiment
  • Cryptocurrency and stock search functionality
  • Sentiment voting on individual cryptocurrency pages
  • Dark mode

Data for cryptocurrency is provided by CoinGecko, and data for stock market is provided by yahoo finance and worldtradingdata.
Link: https://www.marketinfo.io


Title: Re: Crypto News Aggregator
Post by: TimtheYoutuber on November 10, 2019, 04:13:45 PM
These sort of projects are awesome, however, you should use this data on the backend of a different project. This sort of thing has been done before. You should try and implement algorithms that use this data to build helpful graphs or research templates that people could use.



Anywho, I like the project it looks nice :)


Title: Re: Crypto News Aggregator
Post by: hugeblack on November 10, 2019, 05:03:50 PM
Good effort, you need to adjust the static interface to look more modern but I have some questions:

 - Update data: How is the data updated? Is it automatically? How long?
 - Fake News: Lots of fake news, do you have an algorithm to check it?
 - Repeat news or news is not relevant: Is there an algorithm to do it?
 - Charts: Charts seem static and cannot be adjustable.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 11, 2019, 12:51:55 AM
These sort of projects are awesome, however, you should use this data on the backend of a different project. This sort of thing has been done before. You should try and implement algorithms that use this data to build helpful graphs or research templates that people could use.



Anywho, I like the project it looks nice :)

I'm always looking for new projects relating to the cryptocurrency space. If I can think of an idea that uses this data and would be useful to people, I'll be sure to either add the idea to this current project or create a different one altogether.

Thank you :)


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 11, 2019, 01:01:16 AM
Good effort, you need to adjust the static interface to look more modern but I have some questions:

 - Update data: How is the data updated? Is it automatically? How long?
 - Fake News: Lots of fake news, do you have an algorithm to check it?
 - Repeat news or news is not relevant: Is there an algorithm to do it?
 - Charts: Charts seem static and cannot be adjustable.


The interface could definitely look better. Design isn't my best area lol. I'll see what I can do though to make it look more modern.

-News is pulled once an hour and is updated upon any refresh on page. All cryptocurrency market data is updated every minute. Lastly, stock market overview and bitcoin sentiment is updated once a day.
-I currently do not have an algorithm to check fake news. This is a good idea though and something I'm interested in doing. This may take some time to implement.
-I currently do not have an algorithm to check repeat news. This is another good idea and something I should be able to implement fairly easily.
-I designed the charts to be fairly static, however, I'm open to changing how you can interact with them. In what ways would you like them to be adjustable?

Thank you for the feedback :)


Title: Re: Crypto News Aggregator
Post by: Diced90 on November 11, 2019, 05:45:52 AM
Are you aggregating news from crypto news sites? If so, they are quite slow and post latest news that have been reported by other main news portals which are already a few hours old. Newsbtc and SeekingAlpha are big examples of such sites. Sometimes they copy news from Cointelegraph or Coindesk.

Also I would like a real time interface, are you looking to sell this? or just develop on this?


Title: Re: Crypto News Aggregator
Post by: loafof on November 11, 2019, 06:02:41 AM


Great! I think improvements can be made though regarding news filtering. There's just quite a bit of the same news article on different platforms.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 11, 2019, 06:36:25 AM


Great! I think improvements can be made though regarding news filtering. There's just quite a bit of the same news article on different platforms.

Thanks! I agree and will start working on it. Another person above you also mentioned that I should create an algorithm that removes duplicate news articles. Once I get this to work, I'll be sure to update this thread. Thanks for the feedback.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 11, 2019, 06:39:10 AM
Are you aggregating news from crypto news sites? If so, they are quite slow and post latest news that have been reported by other main news portals which are already a few hours old. Newsbtc and SeekingAlpha are big examples of such sites. Sometimes they copy news from Cointelegraph or Coindesk.

Also I would like a real time interface, are you looking to sell this? or just develop on this?

I am aggregating news from crypto news sites. Duplicate news is a problem I am running into and am working on a solution to filter out duplicate news articles. At the moment I'm pulling all feeds once an hour. A real time interface would be nice and might be something I'll look into. For the time being, I am looking to develop on this. Also, thank you for the feedback.


Title: Re: Crypto News Aggregator
Post by: NeuroticFish on November 11, 2019, 01:52:57 PM
Not bad. I like the colors for dark mode better chosen than the ones for light. A resize for the panels could be useful (I prefer to see Monero price and I don't care about Nasdaq and S&P).


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 12, 2019, 07:59:52 AM
Not bad. I like the colors for dark mode better chosen than the ones for light. A resize for the panels could be useful (I prefer to see Monero price and I don't care about Nasdaq and S&P).

Thank you. I'll keep trying different light colors on my end to see if I can find something that looks presentable. The ability to resize the panels is an interesting option. A customizable market overview may be a feature I'd like to add in the near future. This way you can view only what you care about seeing.


Title: Re: Crypto News Aggregator
Post by: Jating on November 12, 2019, 10:29:49 AM
Good effort, you need to adjust the static interface to look more modern but I have some questions:

 - Update data: How is the data updated? Is it automatically? How long?
 - Fake News: Lots of fake news, do you have an algorithm to check it?
 - Repeat news or news is not relevant: Is there an algorithm to do it?
 - Charts: Charts seem static and cannot be adjustable.


The interface could definitely look better. Design isn't my best area lol. I'll see what I can do though to make it look more modern.

-News is pulled once an hour and is updated upon any refresh on page. All cryptocurrency market data is updated every minute. Lastly, stock market overview and bitcoin sentiment is updated once a day.
-I currently do not have an algorithm to check fake news. This is a good idea though and something I'm interested in doing. This may take some time to implement.
-I currently do not have an algorithm to check repeat news. This is another good idea and something I should be able to implement fairly easily.
-I designed the charts to be fairly static, however, I'm open to changing how you can interact with them. In what ways would you like them to be adjustable?

Thank you for the feedback :)

I don't think you can have a algo to check for fake news unless you do it manually. If I remember correctly, in reddit there are a lot of fake news and it can only be tagged by the moderator, meaning their's manual intervention.

Maybe you can add up a filter function though? If we wanted to see just bitcoin news, then perhaps we filter it out? Or at least a category in the side of the news, so that we won't have to go to all of them and we can easily pick the news we wanted to read.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 12, 2019, 11:20:23 AM
Good effort, you need to adjust the static interface to look more modern but I have some questions:

 - Update data: How is the data updated? Is it automatically? How long?
 - Fake News: Lots of fake news, do you have an algorithm to check it?
 - Repeat news or news is not relevant: Is there an algorithm to do it?
 - Charts: Charts seem static and cannot be adjustable.


The interface could definitely look better. Design isn't my best area lol. I'll see what I can do though to make it look more modern.

-News is pulled once an hour and is updated upon any refresh on page. All cryptocurrency market data is updated every minute. Lastly, stock market overview and bitcoin sentiment is updated once a day.
-I currently do not have an algorithm to check fake news. This is a good idea though and something I'm interested in doing. This may take some time to implement.
-I currently do not have an algorithm to check repeat news. This is another good idea and something I should be able to implement fairly easily.
-I designed the charts to be fairly static, however, I'm open to changing how you can interact with them. In what ways would you like them to be adjustable?

Thank you for the feedback :)

I don't think you can have a algo to check for fake news unless you do it manually. If I remember correctly, in reddit there are a lot of fake news and it can only be tagged by the moderator, meaning their's manual intervention.

Maybe you can add up a filter function though? If we wanted to see just bitcoin news, then perhaps we filter it out? Or at least a category in the side of the news, so that we won't have to go to all of them and we can easily pick the news we wanted to read.

I think you're right about fake news. Manually filtering seems like the only viable solution. I did implement a duplicate news filter on my test server but it doesn't detect news that are similar. I'm finding this much more difficult to implement but am still looking into it. As for seeing specific news, you can search at the top right and get specific news i.e. bitcoin news. Also, thanks for the feedback.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 13, 2019, 06:14:24 AM
Quote
Maybe you can add up a filter function though? If we wanted to see just bitcoin news, then perhaps we filter it out? Or at least a category in the side of the news, so that we won't have to go to all of them and we can easily pick the news we wanted to read.

I updated it so where you can use the search bar at the top right to filter out news i.e. just ethereum news. Now it should be easier to see the news you want to read.


Title: Re: Crypto News Aggregator
Post by: Kemarit on November 14, 2019, 07:21:38 AM
Ok, how to the Search function works?

I type "Bitcoin" on the Search texbox and this is the result I got:

Quote
No news to display.

https://www.marketinfo.io/aml-bitcoin

Did I do something wrong here? Or the Search function has some bugs on it?


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 14, 2019, 08:22:19 AM
Ok, how to the Search function works?

I type "Bitcoin" on the Search texbox and this is the result I got:

Quote
No news to display.

https://www.marketinfo.io/aml-bitcoin

Did I do something wrong here? Or the Search function has some bugs on it?

Hello, It looks like you searched for aml-bitcoin which is the first prediction result(I need to fix this). When you type “bitcoin” in the search bar, scroll 1 down in the prediction box underneath and you should see “bitcoin|btc.” Not a bug, but I need to make the search prediction better.
https://www.marketinfo.io/bitcoin is the correct page for bitcoin


Title: Re: Crypto News Aggregator
Post by: Kemarit on November 15, 2019, 01:46:14 AM
Ok, how to the Search function works?

I type "Bitcoin" on the Search texbox and this is the result I got:

Quote
No news to display.

https://www.marketinfo.io/aml-bitcoin

Did I do something wrong here? Or the Search function has some bugs on it?

Hello, It looks like you searched for aml-bitcoin which is the first prediction result(I need to fix this). When you type “bitcoin” in the search bar, scroll 1 down in the prediction box underneath and you should see “bitcoin|btc.” Not a bug, but I need to make the search prediction better.
https://www.marketinfo.io/bitcoin is the correct page for bitcoin


Ok, thanks for clearing that up, but I guess you need to fix it as well. Because as I user, those filters are supposed to give us the best options right?

Anyway, it's good that we have more options, you can also look at (https://faws.com/). I used that one for years now, but as I have said, your project is a good alternative, so keep it up.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 15, 2019, 02:20:42 AM
Quote

Ok, thanks for clearing that up, but I guess you need to fix it as well. Because as I user, those filters are supposed to give us the best options right?

Anyway, it's good that we have more options, you can also look at (https://faws.com/). I used that one for years now, but as I have said, your project is a good alternative, so keep it up.

No problem and thank you for giving me feedback. I fixed the problem. Let me know if you run into any more problems. Also, that's a nice website, I like the up-vote functionality on the news feed.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 17, 2019, 03:18:41 AM
For anyone that may be following this thread:
-I implemented a solution to remove duplicate articles.
-I launched an update that adds the ability to see a preview of each aggregated feed item.
-Added more news sources
If anyone runs into any issues, feel free to let me know. Thanks


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 26, 2019, 11:47:27 AM
I have updated https://www.marketinfo.io (https://www.marketinfo.io) to be more mobile friendly. Details and market overview can now be toggled on mobile. News now has much more viewing space, and I improved the scrolling. If you were previously using the website, you may have to clear your browser cache to view updates. If you run into any issues, feel free to message me. Thanks


Title: Re: Crypto News Aggregator
Post by: Kemarit on November 27, 2019, 08:41:50 AM
I try to used your search function again with this parameters:

(https://www.marketinfo.io/bitcoin)

And I thought that I will just get news related to Bitcoin only. But tt looks like it filtered:

Quote
11-27-2019 Tron’s TRX is Pumping While Crypto Markets are Dumping, But Why?

And then some Bitcoin Cash and Bitcoin SV news as well?


Title: Re: Crypto News Aggregator
Post by: cryptohappening on November 27, 2019, 10:30:16 AM
I try to used your search function again with this parameters:

(https://www.marketinfo.io/bitcoin)

And I thought that I will just get news related to Bitcoin only. But tt looks like it filtered:

Quote
11-27-2019 Tron’s TRX is Pumping While Crypto Markets are Dumping, But Why?

And then some Bitcoin Cash and Bitcoin SV news as well?

Thank you for bringing this to my attention.

Edit: After some work, I was able to fix the problem and improve the search filter further. Let me know if you run into any more issues.

Edit2: Unrelated, but I also made it to where you can click the logo at the top left and it takes you back home(pulling most current data) without having to reload the entire page.


Title: Re: Crypto News Aggregator
Post by: Kemarit on December 02, 2019, 05:21:39 AM
I try to used your search function again with this parameters:

(https://www.marketinfo.io/bitcoin)

And I thought that I will just get news related to Bitcoin only. But tt looks like it filtered:

Quote
11-27-2019 Tron’s TRX is Pumping While Crypto Markets are Dumping, But Why?

And then some Bitcoin Cash and Bitcoin SV news as well?

Thank you for bringing this to my attention.

Edit: After some work, I was able to fix the problem and improve the search filter further. Let me know if you run into any more issues.

Edit2: Unrelated, but I also made it to where you can click the logo at the top left and it takes you back home(pulling most current data) without having to reload the entire page.

Sorry I was out in the last couple of days, wasn't able to reply to you in timely fashion.

Anyways, it looks like it's working as design will try to test it as I have more time today.

Yes, the logo click is very good, at it saves time for user to reload just to go back to the home page.

What's with the eLTC (eltc2) though

$1.73e-06
Market cap: $145.17 #1871
24h volume: $0.0

https://www.marketinfo.io/eltc

Maybe you need some tweak so that the calculations is readable.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on December 02, 2019, 11:26:26 AM
Quote
Sorry I was out in the last couple of days, wasn't able to reply to you in timely fashion.

Anyways, it looks like it's working as design will try to test it as I have more time today.

Yes, the logo click is very good, at it saves time for user to reload just to go back to the home page.

What's with the eLTC (eltc2) though

$1.73e-06
Market cap: $145.17 #1871
24h volume: $0.0

https://www.marketinfo.io/eltc

Maybe you need some tweak so that the calculations is readable.

No problem. I appreciate you taking the time to test out the web application. The very small prices should be more readable now. Sorry for the delay, I had to fix another problem that came up with the news feed. Everything should be in working order now.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on December 09, 2019, 08:17:25 AM
Added sentiment voting on individual cryptocurrency pages. Once more data is available, I'll display the corresponding sentiments after vote is submitted.

Edit: I changed the styling a little in attempt to make the application look more modern.

Edit2: Time since each feed item has been published is now displayed in a more easy to read minutes/hours/days format. Should be easier to see which articles are most recent.


Title: Re: Crypto News Aggregator
Post by: Kemarit on December 11, 2019, 11:47:04 AM
Added sentiment voting on individual cryptocurrency pages. Once more data is available, I'll display the corresponding sentiments after vote is submitted.

I voted, but it seems I'm lost as I can't find where my vote goes.

Edit: I changed the styling a little in attempt to make the application look more modern.

Yes, I see the changes, very nice feel and look of the new website. Keep it up.

Edit2: Time since each feed item has been aggregated is now displayed in a more easy to read minutes/hours/days format. Should be easier to see which articles are most recent.

Good that you implemented this one, was just about to suggest it.


Title: Re: Crypto News Aggregator
Post by: cryptohappening on December 11, 2019, 12:43:19 PM
I voted, but it seems I'm lost as I can't find where my vote goes.

Edit: After voting, you can now see a bar that represents the overall up-votes vs down-votes. Also, thank you for the feedback. Glad you like the changes!


Title: Re: Crypto News Aggregator
Post by: Kemarit on December 15, 2019, 05:49:22 AM
I voted, but it seems I'm lost as I can't find where my vote goes.

Edit: After voting, you can now see a bar that represents the overall up-votes vs down-votes. Also, thank you for the feedback. Glad you like the changes!

Ok, it looks like the vote is working as designed.

And looking at the response, seems to me that Bitcoin enthusiast are still very positive about it eventhough the price doesn't seems to look good at this point.

How many times could be vote though? I refresh the page and you can vote again. Are there no limitations? Or at least 1 vote per 24 hours?


Title: Re: Crypto News Aggregator
Post by: cryptohappening on December 15, 2019, 09:05:39 AM

Ok, it looks like the vote is working as designed.

And looking at the response, seems to me that Bitcoin enthusiast are still very positive about it eventhough the price doesn't seems to look good at this point.

How many times could be vote though? I refresh the page and you can vote again. Are there no limitations? Or at least 1 vote per 24 hours?

At the moment, no limitations due to not having an account login system(I prefer not to have a login system). I could store cookies to try to limit 1 vote per 24 hours but that could still be abused by users clearing their browser cache. I figure there isn't much to gain from users potentially abusing the voting system. Eventually, when there are more votes coming in, I'll probably reset all votes every 24 hours.


Title: Re: Crypto News Aggregator
Post by: Kemarit on December 16, 2019, 10:00:03 AM

Ok, it looks like the vote is working as designed.

And looking at the response, seems to me that Bitcoin enthusiast are still very positive about it eventhough the price doesn't seems to look good at this point.

How many times could be vote though? I refresh the page and you can vote again. Are there no limitations? Or at least 1 vote per 24 hours?

At the moment, no limitations due to not having an account login system(I prefer not to have a login system). I could store cookies to try to limit 1 vote per 24 hours but that could still be abused by users clearing their browser cache. I figure there isn't much to gain from users potentially abusing the voting system. Eventually, when there are more votes coming in, I'll probably reset all votes every 24 hours.

OK it makes sense. I just thought of when trying to test the new voting system and thinking that somewhere along the line it could be abuse.

Keep up the good work though, you just don't know, maybe down the line you will attract potential users of your system so you need to keep it updated.