Bitcoin Forum
December 05, 2016, 10:41:14 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: WTB: Script to parse website and create a table & text formatted for forum  (Read 2278 times)
Cablesaurus
Sr. Member
****
Offline Offline

Activity: 302



View Profile WWW
July 05, 2011, 06:00:25 AM
 #1

I'm looking to find someone who can produce a simple clean script to parse item pictures/descriptions/links from my web site.

I'd like this to format them automatically into a clean table with description info and txt/image links to back to the site.

A bonus would be if this could also include the eBay links which seem to ever constantly change. I'd like to aim to also have a table in which can interlink eBay items produced with a variation of this generator as well.

PCIe Extender Cables; Dummy Plugs, Fans; PSU Cables; Cases & More
Visit www.Cablesaurus.com and our forum thread at http://bitcointalk.org/index.php?topic=6128.0
1480934474
Hero Member
*
Offline Offline

Posts: 1480934474

View Profile Personal Message (Offline)

Ignore
1480934474
Reply with quote  #2

1480934474
Report to moderator
1480934474
Hero Member
*
Offline Offline

Posts: 1480934474

View Profile Personal Message (Offline)

Ignore
1480934474
Reply with quote  #2

1480934474
Report to moderator
There are several different types of Bitcoin clients. Header-only clients like MultiBit trust that the majority of mining power is honest for the purposes of enforcing network rules such as the 21 million BTC limit. Full clients do not trust miners in this way.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480934474
Hero Member
*
Offline Offline

Posts: 1480934474

View Profile Personal Message (Offline)

Ignore
1480934474
Reply with quote  #2

1480934474
Report to moderator
1480934474
Hero Member
*
Offline Offline

Posts: 1480934474

View Profile Personal Message (Offline)

Ignore
1480934474
Reply with quote  #2

1480934474
Report to moderator
1480934474
Hero Member
*
Offline Offline

Posts: 1480934474

View Profile Personal Message (Offline)

Ignore
1480934474
Reply with quote  #2

1480934474
Report to moderator
error
Hero Member
*****
Offline Offline

Activity: 574



View Profile
July 05, 2011, 06:11:24 AM
 #2

It could be done, but it would be easier to just read all the relevant information out of your database.

15UFyv6kfWgq83Pp3yhXPr8rknv9m6581W
Cablesaurus
Sr. Member
****
Offline Offline

Activity: 302



View Profile WWW
July 05, 2011, 06:12:53 AM
 #3

It could be done, but it would be easier to just read all the relevant information out of your database.

If someone is familiar enough with the zencart db structure that's one thing. I don't have a lot of time to set this up though which is why I'd like to outsource the job in such a way no internal access is needed.

PCIe Extender Cables; Dummy Plugs, Fans; PSU Cables; Cases & More
Visit www.Cablesaurus.com and our forum thread at http://bitcointalk.org/index.php?topic=6128.0
error
Hero Member
*****
Offline Offline

Activity: 574



View Profile
July 05, 2011, 06:14:54 AM
 #4

It could be done, but it would be easier to just read all the relevant information out of your database.

If someone is familiar enough with the zencart db structure that's one thing. I don't have a lot of time to set this up though which is why I'd like to outsource the job in such a way no internal access is needed.

Well, your description isn't detailed enough yet to have any idea of what it is you want to build. Can you provide some better idea of what you want to end up with?

15UFyv6kfWgq83Pp3yhXPr8rknv9m6581W
Cablesaurus
Sr. Member
****
Offline Offline

Activity: 302



View Profile WWW
July 05, 2011, 06:19:57 AM
 #5

On the front page of http://forum.bitcoin.org/index.php?topic=6128.0

You'll see I have listings such as:
Quote

As more products are added and the pictures and descriptions occasionally changed this gets a bit hard to keep up with, and it's useful information for those coming from the forum. Additionally with eBay item limits restrictions, items will sell out and I have to shuffle things around and re-list, causing the eBay links to change.

In the end I'd like something very simple that collects this information from the product listings on my main site such as https://cablesaurus.com/index.php?main_page=product_info&cPath=1&products_id=8 and outputs it into a simple table with text, which I can run when an update is needed, and take the generated code and plug it into my eBay items and the forum post manually. (This may have to be done with 2 variant of a similar include code generator script.)


PCIe Extender Cables; Dummy Plugs, Fans; PSU Cables; Cases & More
Visit www.Cablesaurus.com and our forum thread at http://bitcointalk.org/index.php?topic=6128.0
error
Hero Member
*****
Offline Offline

Activity: 574



View Profile
July 05, 2011, 06:23:48 AM
 #6

On the front page of http://forum.bitcoin.org/index.php?topic=6128.0

You'll see I have listings such as:
Quote

As more products are added and the pictures and descriptions occasionally changed this gets a bit hard to keep up with, and it's useful information for those coming from the forum. Additionally with eBay item limits restrictions, items will sell out and I have to shuffle things around and re-list, causing the eBay links to change.

In the end I'd like something very simple that collects this information and outputs it into a simple table with text, which I can run when an update is needed, and take the generated code and plug it into my eBay items and the forum post manually.

OK, now we're getting somewhere.

So, to be clear, you're maintaining your inventory, pictures, descriptions, etc., in Zen Cart, and you want to export this data so you can post it here on the Bitcoin forum and on eBay?

15UFyv6kfWgq83Pp3yhXPr8rknv9m6581W
Cablesaurus
Sr. Member
****
Offline Offline

Activity: 302



View Profile WWW
July 05, 2011, 06:25:27 AM
 #7

Yep, exactly.

The Bitcoin version would include images, descriptions, and 2 links per item.

The eBay version would likely just need to be a table of images with product title and no detailed description info, just a direct link via the image to the other eBay item sale pages.

PCIe Extender Cables; Dummy Plugs, Fans; PSU Cables; Cases & More
Visit www.Cablesaurus.com and our forum thread at http://bitcointalk.org/index.php?topic=6128.0
alkhdaniel
Member
**
Offline Offline

Activity: 114


View Profile
July 05, 2011, 08:39:52 AM
 #8

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases


like this? (note: takes ~15-20s to load)
Deaddy
Newbie
*
Offline Offline

Activity: 17


View Profile WWW
July 06, 2011, 04:36:03 AM
 #9

Hi,

I would write a python-script (python 2.6 or higher) depending on BeautifulSoup, which would crawl through the categories listet on top of your Mainpage, generating a list for each category.
Then I would make a simple table and provide an interface, so you would later be able to create your own templates or modify existing ones with the variables like $category, $img, $productname, $description, $shoplink and $price (guess these were all?). However, I can't provide support on Windows and I would prefer it to release the script to public domain afterwards.
I'm charging a rate of 20€/2 BTC per started 30 minutes, however I'd cap it at 2 hours for the initial coding, so you would probably pay 4-8 BTC. If you wish I can also send you an invoice in Euro per E-Mail so you could deduce it from taxes.

13rigybYMphatCxAhKksDeozbWE1s6sP6L
nhodges
Sr. Member
****
Offline Offline

Activity: 308


View Profile
July 06, 2011, 05:05:24 PM
 #10

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases


like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/

alkhdaniel
Member
**
Offline Offline

Activity: 114


View Profile
July 06, 2011, 05:15:22 PM
 #11

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases


like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/
Not seeing why this would be necessary since the script would only be ran when he needs to update the thread (meaning he has changed description on an item, thus making the cache useless). The page is not supposed to be public for anyone to view, i just put it up as proof of work (and to see if this is what he wanted).

Or did i miss something?
nhodges
Sr. Member
****
Offline Offline

Activity: 308


View Profile
July 06, 2011, 05:26:24 PM
 #12

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases


like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/
Not seeing why this would be necessary since the script would only be ran when he needs to update the thread (meaning he has changed description on an item, thus making the cache useless). The page is not supposed to be public for anyone to view, i just put it up as proof of work (and to see if this is what he wanted).

Or did i miss something?

He could run it on a cron and it would be instantly available for him, instead of having to wait 15-20s each time, procing the script.

alkhdaniel
Member
**
Offline Offline

Activity: 114


View Profile
July 06, 2011, 05:48:40 PM
 #13

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases


like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/
Not seeing why this would be necessary since the script would only be ran when he needs to update the thread (meaning he has changed description on an item, thus making the cache useless). The page is not supposed to be public for anyone to view, i just put it up as proof of work (and to see if this is what he wanted).

Or did i miss something?

He could run it on a cron and it would be instantly available for him, instead of having to wait 15-20s each time, procing the script.
But it would either be not up to date information unless he runs it about once a minute (and then he would have to wait 15-20s every 4th time anyway, and it would take up MUCH more resources). The script is for getting the most up to date information of product information, not yesterdays information. Also if the script was run locally it probaly would not take 15-20s, not that waiting 15-20s should be a problem since you can just prepare to edit the old post during that timeframe or simply wait (seriously, its less than 1/4th of a minute of WAITING time compared to 15-30 minutes of parsing it manually).
Also if having the information in less than 15s is critical you could just save everything in a .txt every time the script is run instead of playing around with caches, this would still require using up more resources and probaly get outdated information.

I might still be misunderstanding something though, not sure..?
Deaddy
Newbie
*
Offline Offline

Activity: 17


View Profile WWW
July 06, 2011, 06:40:53 PM
 #14

Well, PHP might be slow hell, but I guess most of the time is still transferring the data. If he'd run it directly on the server the runtime wouldn't be noticable. Wink

13rigybYMphatCxAhKksDeozbWE1s6sP6L
nhodges
Sr. Member
****
Offline Offline

Activity: 308


View Profile
July 06, 2011, 11:04:03 PM
 #15

Well, PHP might be slow hell, but I guess most of the time is still transferring the data. If he'd run it directly on the server the runtime wouldn't be noticable. Wink

Your statement is "mind bottling" sir ...

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases


like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/
Not seeing why this would be necessary since the script would only be ran when he needs to update the thread (meaning he has changed description on an item, thus making the cache useless). The page is not supposed to be public for anyone to view, i just put it up as proof of work (and to see if this is what he wanted).

Or did i miss something?

He could run it on a cron and it would be instantly available for him, instead of having to wait 15-20s each time, procing the script.
But it would either be not up to date information unless he runs it about once a minute (and then he would have to wait 15-20s every 4th time anyway, and it would take up MUCH more resources). The script is for getting the most up to date information of product information, not yesterdays information. Also if the script was run locally it probaly would not take 15-20s, not that waiting 15-20s should be a problem since you can just prepare to edit the old post during that timeframe or simply wait (seriously, its less than 1/4th of a minute of WAITING time compared to 15-30 minutes of parsing it manually).
Also if having the information in less than 15s is critical you could just save everything in a .txt every time the script is run instead of playing around with caches, this would still require using up more resources and probaly get outdated information.

I might still be misunderstanding something though, not sure..?

I guess you're right, I just see this as a very valuable promotional tool (externally) that he could use to give his own visitors the ability promote his products on websites they frequent. In the way you designed it originally, it would be very easy to just call your script with the item id number and display a textbox with the bbcode formatted links for a user to copy/paste into the forum of their choice.

Cablesaurus
Sr. Member
****
Offline Offline

Activity: 302



View Profile WWW
July 07, 2011, 10:40:58 AM
 #16

Quote
I guess you're right, I just see this as a very valuable promotional tool (externally) that he could use to give his own visitors the ability promote his products on websites they frequent. In the way you designed it originally, it would be very easy to just call your script with the item id number and display a textbox with the bbcode formatted links for a user to copy/paste into the forum of their choice.

This would definitely be a bonus. I hadn't originally thought of such a setup, but it'd be of great use.

PCIe Extender Cables; Dummy Plugs, Fans; PSU Cables; Cases & More
Visit www.Cablesaurus.com and our forum thread at http://bitcointalk.org/index.php?topic=6128.0
alkhdaniel
Member
**
Offline Offline

Activity: 114


View Profile
July 07, 2011, 10:55:18 AM
 #17

Quote
I guess you're right, I just see this as a very valuable promotional tool (externally) that he could use to give his own visitors the ability promote his products on websites they frequent. In the way you designed it originally, it would be very easy to just call your script with the item id number and display a textbox with the bbcode formatted links for a user to copy/paste into the forum of their choice.

This would definitely be a bonus. I hadn't originally thought of such a setup, but it'd be of great use.
This can do that.

Also is this what you originally looked for? (~15s load time)

Please reply or post here if that's what you want and/or you want to change something, can throw in the first script for free.
Cablesaurus
Sr. Member
****
Offline Offline

Activity: 302



View Profile WWW
July 11, 2011, 08:42:36 PM
 #18

pmed

PCIe Extender Cables; Dummy Plugs, Fans; PSU Cables; Cases & More
Visit www.Cablesaurus.com and our forum thread at http://bitcointalk.org/index.php?topic=6128.0
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!