[10 BTC reward *paid*] Guide to install DSpace on Amazon EC2

[Sukrim]

As I'm currently struggling with a task for a project of mine and don't want to waste any more time on fighting the Amazon Linux command line, here's my deal:

As soon as I get from you:

* A step-by-step/command-by-command instruction how to create a running DSpace installation on a Amzon EC2 Micro instance using a fresh, naked + empty 32 or 64 bit Amazon Linux AMI with SSH access and assuming "ec2-user" just being logged into a virgin machine for the first time as the starting point. This guide should be posted here in the open, in case you think it might need some polishing but you don't want potential competitors copying your work, you can also send me links to google docs or so via PM - after the task is done however, I will publish the guide here and hopefully also on the DSpace Wiki.
* A bitcoin payout address

until 15.4.2012 (23:59:59 GMT... )

I will send you 10 Bitcoins to that address. If the guide is really well written and/or the task is really quickly fulfilled (like still today for example) and/or also takes care of potential risks for live deployment (something like "choose a good password here, this can be accessed via a web interface too!"), I also consider a bonus.

I reserve the right to first try out your solution, also please keep in mind that just writing "do this, that and then follow the main guide" is NOT enough for me. Basically I want to be able to put every command one after the other just in a shellscript and after that script is done, DSpace is running (except for some changes with vim/nano in config files or so - but I think you get the idea). As a rule of thumb, no "enter the basic config information and continue" - explicitly write what goes in which config file.

About the software to be used:
DSpace: http://www.dspace.org/
Amazon EC2: http://aws.amazon.com/en/ec2/
Besides setting up DSpace, you'll likely also need Apache Tomcat, Apache Maven, Apache Ant and a Postgres-SQL database set up with at least some basic settings, so it is possible to run/access the DSpace software package.

Some starting points:
DSpace Deployment on EC2:
https://wiki.duraspace.org/display/GSOC/Amazon+EC2+DSpace+Deployment (<-- an example for a "guide" that is NOT enough... )
General DSpace installation guide:
https://wiki.duraspace.org/display/DSDOC18/Installation
Amazon Linux is by the way generally quite similar to RedHat/CentOS, in case you're wondering. Also if you don't have access to Amazon EC2 yet (but are a better Linux admin than me) and want to do this, there's a whole year of free operation included in Amazon's services, so you likely won't have any costs besides your own time doing this task.

Any questions? Feel free to ask, preferrably here in the thread.

[1715165412]

1715165412

[1715165412]

1715165412

[1715165412]

1715165412

[BinaryMage]

There's a prebuilt Amazon EC2 image here, would that suit your needs? If you're looking for easy, that's pretty much the paragon. (1Ly7emr4pYT5UjKZ4VWEjbym2s5F9oA7y8)

[Andrey]

Please confirm if you still need instructions, I am about to compile them

[Sukrim]

There's a prebuilt Amazon EC2 image here, would that suit your needs? If you're looking for easy, that's pretty much the paragon. (1Ly7emr4pYT5UjKZ4VWEjbym2s5F9oA7y8)

I'm sorry, but as far as I understand it, you have to pay Jumpbox additionally (60 USD/month, a multiple of what I estimate to pay to Amazon...) just to be able/allowed to use this image on EC2. Also it uses Ubuntu 10.04, not Amazon Linux. It might help people to figure out how to install DSpace though I guess. I knew already before about this service but decided against it. Nevertheless thanks for your time and your post!

I still need instructions @Andrey.

[BinaryMage]

There's a prebuilt Amazon EC2 image here, would that suit your needs? If you're looking for easy, that's pretty much the paragon. (1Ly7emr4pYT5UjKZ4VWEjbym2s5F9oA7y8)

I'm sorry, but as far as I understand it, you have to pay Jumpbox additionally (60 USD/month, a multiple of what I estimate to pay to Amazon...) just to be able/allowed to use this image on EC2. Also it uses Ubuntu 10.04, not Amazon Linux. It might help people to figure out how to install DSpace though I guess. I knew already before about this service but decided against it. Nevertheless thanks for your time and your post!

I still need instructions @Andrey.

My bad, didn't read the fine print. If Andrey doesn't come up with anything for you by tomorrow, I'll put something together.

[Andrey]

My bad, didn't read the fine print. If Andrey doesn't come up with anything for you by tomorrow, I'll put something together.

Probably I wont, I haven't used my AWS for some time and it is now blocked. Don't want to provide instructions which I don't check personally. So you have time. I contacted the support, but not sure when they do respond, so you probably go ahead, no need for delay in such a simple task.

[BinaryMage]

My bad, didn't read the fine print. If Andrey doesn't come up with anything for you by tomorrow, I'll put something together.

Probably I wont, I haven't used my AWS for some time and it is now blocked. Don't want to provide instructions which I don't check personally. So you have time. I contacted the support, but not sure when they do respond, so you probably go ahead, no need for delay in such a simple task.

Blech, didn't realize they required a credit card to sign up. (Don't want Amazon to have that) I could put something theoretical together, but I won't be able to test it...

Hopefully someone sees this who actually has an EC2 instance.

[Andrey]

I think on Monday or Tuesday I will have my account so will be able to do that, if still needed by that date.

[Sukrim]

Andrey, yes I could use it on Monday or Tuesday too.I just didn't want to give a date 3 weeks away or so, as it is quite urgent to me. I also decided I want to have quality information fast, which is why I chose to hand out these ~45-50 USD for a task that might be simple for some people more familiar to AWS than me.

If someone can do it faster in good quality though I would prefer that of course!

[paraipan]

I will give it a try. Allow me a few hours.

edit: i have a question for you, what problems do you encounter when following the already existent install instructions ?

https://wiki.duraspace.org/display/DSDOC18/Installation#Installation-Installation

edit2: launch a free bitnami LAPPstack instance and start building your DSpace app from that. I can give you any kind of assistance if needed.

[Sukrim]

edit: i have a question for you, what problems do you encounter when following the already existent install instructions ?

https://wiki.duraspace.org/display/DSDOC18/Installation#Installation-Installation

I had problems connecting to the database and setting it up mostly. Up to Maven it kinda worked but Ant afterwards complained quite quickly that the database connection failed.
Also I wasn't quite sure if I now still need a "dspace" user account or not after reading and trying to follow the wiki page from the GSOC (first link). Also "createuser -a ec2-user -U postgres" won't even work since "createuser" has no command line argument named "-a"...

edit2: launch a free bitnami LAPPstack instance and start building your DSpace app from that. I can give you any kind of assistance if needed.

Here as well: I do not want to use Ubuntu as Server OS, also I'm not really comfortable with using 3rd party AMIs. Additionally I guess I'd still need to install Tomcat there but I won't need Apache httpd and PHP...

[paraipan]

edit: i have a question for you, what problems do you encounter when following the already existent install instructions ?

https://wiki.duraspace.org/display/DSDOC18/Installation#Installation-Installation

I had problems connecting to the database and setting it up mostly. Up to Maven it kinda worked but Ant afterwards complained quite quickly that the database connection failed.
Also I wasn't quite sure if I now still need a "dspace" user account or not after reading and trying to follow the wiki page from the GSOC (first link). Also "createuser -a ec2-user -U postgres" won't even work since "createuser" has no command line argument named "-a"...

So you had some problems dealing with postgres. Try issuing "createuser -U <privileged db account>" and you should be able to create the "dspace" account interactively.

edit2: launch a free bitnami LAPPstack instance and start building your DSpace app from that. I can give you any kind of assistance if needed.

Here as well: I do not want to use Ubuntu as Server OS, also I'm not really comfortable with using 3rd party AMIs. Additionally I guess I'd still need to install Tomcat there but I won't need Apache httpd and PHP...

What distro have you decided using ? Installing all the needed parts to have a working environment depends mostly on the present OS.

[Sukrim]

What distro have you decided using ? Installing all the needed parts to have a working environment depends mostly on the present OS.

See initial post:

[...] a fresh, naked + empty 32 or 64 bit Amazon Linux AMI[...]Amazon Linux is by the way generally quite similar to RedHat/CentOS, in case you're wondering.[...]

Hope this helps.

[paraipan]

What distro have you decided using ? Installing all the needed parts to have a working environment depends mostly on the present OS.

See initial post:

[...] a fresh, naked + empty 32 or 64 bit Amazon Linux AMI[...]Amazon Linux is by the way generally quite similar to RedHat/CentOS, in case you're wondering.[...]

Hope this helps.

ok, got it, sorry you had to repeat yourself. I can help you with an 32 or 64 bit Lucid Lynx, the distro i have experience with.

[Andrey]

I really hope it is not too late, I just restored access to my AWS account and is getting through installing DSpace already. I got past createuser issue you had, but thats not the only one missed from the guides

[Andrey]

Installed and run the dspace, what a mess with those guides, thought it were easy money need few minutes to grab all the commands I put in and double check it on fresh install, hope you still need it.

[Andrey]

After you setup the ec2 instance with Amazon Linux AMI and login into shell do the following commands:

sudo yum update
sudo yum install tomcat6 postgresql postgresql-server svn
mkdir archive
cd archive
wget http://apache.osuosl.org//maven/binaries/apache-maven-3.0.4-bin.zip
sudo unzip apache-maven-3.0.4-bin.zip -d /usr/local/bin
wget http://mirrors.axint.net/apache//ant/binaries/apache-ant-1.8.3-bin.zip
sudo unzip apache-ant-1.8.3-bin.zip -d /usr/local/bin
sudo mkdir /pgdata
sudo chown -R postgres:postgres /pgdata
sudo su postgres
pg_ctl init -D /pgdata
pg_ctl start -D /pgdata
createuser dspace -S -D -R
createdb dspace
<ctrl+D>
sudo useradd -m dspace
sudo mkdir /dspace
sudo chown -R dspace:dspace /dspace/
sudo su - dspace
wget http://sourceforge.net/projects/dspace/files/DSpace%20Stable/1.8.2/dspace-1.8.2-release.zip/download
unzip dspace-1.8.2-release.zip
cd dspace-1.8.2-release/dspace
/usr/local/bin/apache-maven-3.0.4/bin/mvn package
cd target/dspace-1.8.2-build/
/usr/local/bin/apache-ant-1.8.3/bin/ant fresh_install
<ctrl+D>
sudo cp -r /dspace/webapps/* /usr/share/tomcat6/webapps/
sudo /etc/init.d/tomcat6 start
sudo /dspace/bin/dspace create-administrator

set the AWS security for the host to have port 8080 open
navigate to the <aws instance address>:8080/xmlui of the newly created DSpace!

My BTC purse is 1GNmnPf67wj72rTPF8GLz6jS7T354MVnbg

[Sukrim]

Sounds and looks great, I'll check it out and as soon as I can confirm it, the money is yours (it's kinda late at night now at my place, so I'll do it tomorrow morning)!

Man, I'm so glad I can finally move on with that project, being stuck on _installation_ sucked big time!

[Andrey]

Great news that you still needed it. It was about 1:00 when I finished
The guide can be optimized, with some steps eliminated, cause I started from using available guides and modified it to work. But as you need a working guide and you are not going to install it everyday I think this one will do. But if you want it polished please let me know.

[Sukrim]

at "pg_ctl start -D /pgdata" I get:

bash-4.1$ pg_ctl start -D /pgdata
could not change directory to "/home/ec2-user/archive"
server starting

Seems to work nevertheless though.

Another thing to keep in mind for anyone trying this out is that the maven assembly seems to take ages(!) sometimes, make sure your SSH connections don't time out.

directly after running ant I get the following error though:

Code:

test_database:
     [java] 2012-04-19 14:55:54,068 INFO  org.dspace.core.ConfigurationManager @ Loading system provided config property (-Ddspace.configuration): config/dspace.cfg
     [java] 2012-04-19 14:55:54,085 INFO  org.dspace.core.ConfigurationManager @ Using default log4j provided log configuration,if unintended, check your dspace.cfg for (log.init.config)
     [java]
     [java] Attempting to connect to database:
     [java]  - URL: jdbc:postgresql://localhost:5432/dspace
     [java]  - Driver: org.postgresql.Driver
     [java]  - Username: dspace
     [java]  - Password: dspace
     [java]  - Schema: null
     [java]
     [java] Testing connection...
     [java]
     [java] Error:
     [java]  - org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
     [java]
     [java] Please see the DSpace documentation for assistance.
     [java]

BUILD FAILED

I tried opening port 5432 in the security settings of the AMI, still no success though...