WTB: Script to parse website and create a table & text formatted for forum

[Cablesaurus]

I'm looking to find someone who can produce a simple clean script to parse item pictures/descriptions/links from my web site.

I'd like this to format them automatically into a clean table with description info and txt/image links to back to the site.

A bonus would be if this could also include the eBay links which seem to ever constantly change. I'd like to aim to also have a table in which can interlink eBay items produced with a variation of this generator as well.

[error]

It could be done, but it would be easier to just read all the relevant information out of your database.

[Cablesaurus]

It could be done, but it would be easier to just read all the relevant information out of your database.

If someone is familiar enough with the zencart db structure that's one thing. I don't have a lot of time to set this up though which is why I'd like to outsource the job in such a way no internal access is needed.

[error]

It could be done, but it would be easier to just read all the relevant information out of your database.

If someone is familiar enough with the zencart db structure that's one thing. I don't have a lot of time to set this up though which is why I'd like to outsource the job in such a way no internal access is needed.

Well, your description isn't detailed enough yet to have any idea of what it is you want to build. Can you provide some better idea of what you want to end up with?

[Cablesaurus]

On the front page of http://forum.bitcoin.org/index.php?topic=6128.0

You'll see I have listings such as:

PCIe x1 Extender Cable:
Bitcoin, Dwolla & Paypal Store Link: https://cablesaurus.com/index.php?main_page=product_info&cPath=1&products_id=8
eBay Store Link: http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=150620070493&ssPageName=STRK:MESELX:IT


PCIe x16 Extender Cable:
Bitcoin, Dwolla & Paypal Store Link: https://cablesaurus.com/index.php?main_page=product_info&cPath=1&products_id=9
eBay Store Link: http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=150621169845&ssPageName=STRK:MESELX:IT

As more products are added and the pictures and descriptions occasionally changed this gets a bit hard to keep up with, and it's useful information for those coming from the forum. Additionally with eBay item limits restrictions, items will sell out and I have to shuffle things around and re-list, causing the eBay links to change.

In the end I'd like something very simple that collects this information from the product listings on my main site such as https://cablesaurus.com/index.php?main_page=product_info&cPath=1&products_id=8 and outputs it into a simple table with text, which I can run when an update is needed, and take the generated code and plug it into my eBay items and the forum post manually. (This may have to be done with 2 variant of a similar include code generator script.)

[error]

On the front page of http://forum.bitcoin.org/index.php?topic=6128.0

You'll see I have listings such as:

PCIe x1 Extender Cable:
Bitcoin, Dwolla & Paypal Store Link: https://cablesaurus.com/index.php?main_page=product_info&cPath=1&products_id=8
eBay Store Link: http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=150620070493&ssPageName=STRK:MESELX:IT


PCIe x16 Extender Cable:
Bitcoin, Dwolla & Paypal Store Link: https://cablesaurus.com/index.php?main_page=product_info&cPath=1&products_id=9
eBay Store Link: http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=150621169845&ssPageName=STRK:MESELX:IT

As more products are added and the pictures and descriptions occasionally changed this gets a bit hard to keep up with, and it's useful information for those coming from the forum. Additionally with eBay item limits restrictions, items will sell out and I have to shuffle things around and re-list, causing the eBay links to change.

In the end I'd like something very simple that collects this information and outputs it into a simple table with text, which I can run when an update is needed, and take the generated code and plug it into my eBay items and the forum post manually.

OK, now we're getting somewhere.

So, to be clear, you're maintaining your inventory, pictures, descriptions, etc., in Zen Cart, and you want to export this data so you can post it here on the Bitcoin forum and on eBay?

[Cablesaurus]

Yep, exactly.

The Bitcoin version would include images, descriptions, and 2 links per item.

The eBay version would likely just need to be a table of images with product title and no detailed description info, just a direct link via the image to the other eBay item sale pages.

[alkhdaniel]

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases

like this? (note: takes ~15-20s to load)

[Deaddy]

Hi,

I would write a python-script (python 2.6 or higher) depending on BeautifulSoup, which would crawl through the categories listet on top of your Mainpage, generating a list for each category.
Then I would make a simple table and provide an interface, so you would later be able to create your own templates or modify existing ones with the variables like $category, $img, $productname, $description, $shoplink and $price (guess these were all?). However, I can't provide support on Windows and I would prefer it to release the script to public domain afterwards.
I'm charging a rate of 20€/2 BTC per started 30 minutes, however I'd cap it at 2 hours for the initial coding, so you would probably pay 4-8 BTC. If you wish I can also send you an invoice in Euro per E-Mail so you could deduce it from taxes.

[nhodges]

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases

like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/

[alkhdaniel]

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases

like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/

Not seeing why this would be necessary since the script would only be ran when he needs to update the thread (meaning he has changed description on an item, thus making the cache useless). The page is not supposed to be public for anyone to view, i just put it up as proof of work (and to see if this is what he wanted).

Or did i miss something?

[nhodges]

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases

like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/

Not seeing why this would be necessary since the script would only be ran when he needs to update the thread (meaning he has changed description on an item, thus making the cache useless). The page is not supposed to be public for anyone to view, i just put it up as proof of work (and to see if this is what he wanted).

Or did i miss something?

He could run it on a cron and it would be instantly available for him, instead of having to wait 15-20s each time, procing the script.

[alkhdaniel]

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases

like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/

Not seeing why this would be necessary since the script would only be ran when he needs to update the thread (meaning he has changed description on an item, thus making the cache useless). The page is not supposed to be public for anyone to view, i just put it up as proof of work (and to see if this is what he wanted).

Or did i miss something?

He could run it on a cron and it would be instantly available for him, instead of having to wait 15-20s each time, procing the script.

But it would either be not up to date information unless he runs it about once a minute (and then he would have to wait 15-20s every 4th time anyway, and it would take up MUCH more resources). The script is for getting the most up to date information of product information, not yesterdays information. Also if the script was run locally it probaly would not take 15-20s, not that waiting 15-20s should be a problem since you can just prepare to edit the old post during that timeframe or simply wait (seriously, its less than 1/4th of a minute of WAITING time compared to 15-30 minutes of parsing it manually).
Also if having the information in less than 15s is critical you could just save everything in a .txt every time the script is run instead of playing around with caches, this would still require using up more resources and probaly get outdated information.

I might still be misunderstanding something though, not sure..?

[Deaddy]

Well, PHP might be slow hell, but I guess most of the time is still transferring the data. If he'd run it directly on the server the runtime wouldn't be noticable.

[nhodges]

Well, PHP might be slow hell, but I guess most of the time is still transferring the data. If he'd run it directly on the server the runtime wouldn't be noticable.

Your statement is "mind bottling" sir ...

Something like http://91.215.158.89/~twyxgiv/cablesaurus/cable.php?id=10 ? (change the id=10 to the id of something in your shop to confirm that it actually works)

Can change it to loop through all id's and parse it a little nicer (so it fits this forum better) and sell the script to you if this is what you're interested in.


edit:
Parsed it to make it possible to copypaste into forum without editing anything, example of id=10:
PCIe x1 -> x16 Adapter Extender Cable
$14.45 or B1.13

* Brand New
* x1 to x16 slot adapter
* Adapter is x1 socket modified to fit larger cards
* Great for intense gaming & Bitcoin mining
* Works for all PCIe bus versions
* 19cm maximum length extension for graphics cards & sound cards
* Ensure proper airflow in your system by creating more room in tight-fit cases

like this? (note: takes ~15-20s to load)

Meh? http://www.snipe.net/2009/03/quick-and-dirty-php-caching/

Not seeing why this would be necessary since the script would only be ran when he needs to update the thread (meaning he has changed description on an item, thus making the cache useless). The page is not supposed to be public for anyone to view, i just put it up as proof of work (and to see if this is what he wanted).

Or did i miss something?

He could run it on a cron and it would be instantly available for him, instead of having to wait 15-20s each time, procing the script.

But it would either be not up to date information unless he runs it about once a minute (and then he would have to wait 15-20s every 4th time anyway, and it would take up MUCH more resources). The script is for getting the most up to date information of product information, not yesterdays information. Also if the script was run locally it probaly would not take 15-20s, not that waiting 15-20s should be a problem since you can just prepare to edit the old post during that timeframe or simply wait (seriously, its less than 1/4th of a minute of WAITING time compared to 15-30 minutes of parsing it manually).
Also if having the information in less than 15s is critical you could just save everything in a .txt every time the script is run instead of playing around with caches, this would still require using up more resources and probaly get outdated information.

I might still be misunderstanding something though, not sure..?

I guess you're right, I just see this as a very valuable promotional tool (externally) that he could use to give his own visitors the ability promote his products on websites they frequent. In the way you designed it originally, it would be very easy to just call your script with the item id number and display a textbox with the bbcode formatted links for a user to copy/paste into the forum of their choice.

[Cablesaurus]

I guess you're right, I just see this as a very valuable promotional tool (externally) that he could use to give his own visitors the ability promote his products on websites they frequent. In the way you designed it originally, it would be very easy to just call your script with the item id number and display a textbox with the bbcode formatted links for a user to copy/paste into the forum of their choice.

This would definitely be a bonus. I hadn't originally thought of such a setup, but it'd be of great use.

[alkhdaniel]

I guess you're right, I just see this as a very valuable promotional tool (externally) that he could use to give his own visitors the ability promote his products on websites they frequent. In the way you designed it originally, it would be very easy to just call your script with the item id number and display a textbox with the bbcode formatted links for a user to copy/paste into the forum of their choice.

This would definitely be a bonus. I hadn't originally thought of such a setup, but it'd be of great use.

This can do that.

Also is this what you originally looked for? (~15s load time)

Please reply or post here if that's what you want and/or you want to change something, can throw in the first script for free.

[Cablesaurus]

pmed