Porting Bitcoind to OpenBSD

[sn-2011a]

I am working on porting bitcoind to OpenBSD. I have run into an issue compiling the wxWidgets dependency.
Is it possible to exclude this dependency as mentioned here: http://bitcointalk.org/index.php?topic=576.0 ?

[indicasteve]

IDk...  but there must be a way.  This thread says there is already a port of bitcoind for OpenBSD:
https://bitcointalk.org/index.php?topic=25501.0

Maybe send a post to the OP and see where you can get it.

I also heard Tom Williams also exclusively used OpenBSD.  There's a few people looking for him.  If you find him, let us know. 

[sn-2011a]

Indicasteve,
Thanks for the reply. The post u reference is for FreeBSD not OpenBSD, but thanks anyways.
Who is Tom?

[twobits]

There are two targets in the makefile, bitcoin (the default, with gui) and bitcoind.  are you doing just a 'make' and not 'make bitcoind'.  You want the latter to not use the wx stuff.

[sn-2011a]

Thank you for your replies. I have gotten past excluding the wXWidgets dep by issueing the `gmake -g makefile.unix bitcoind` command.

After excluding SO_NOSIGPIPE and ignoring MSG_NOSIGNAL in net.cpp and irc.cpp i got to sha256.cpp.

Now, i have a problem with xmmintrin.h in sha256.cpp. xmmintrin supports Intel Streaming SIMD Extensions technology. SSE Intrinsics allow the use of SSE instructions directly from C++ code, without writing the Assembly instructions.

How can i get this to work on my sparc-64, OpenBSD4.9 box?
Output below:

# gmake -f Makefile2 bitcoind
g++ -c -O2 -fstack-protector -Wno-invalid-offsetof -Wformat -g -D__WXDEBUG__ -D__WXGTK__ -DNOPCH -DFOURWAYSSE2 -DUSE_SSL -D__BSD__ -I"/home/root2/bitcoind/deps/include" -I"/usr/local/lib/wx/include/gtk2-unicode-release-2.9" -I"/usr/local/include/db47" -I"/usr/local/include" -L"/usr/local/lib/db47" -L"/home/root2/bitcoind/deps/lib" -L"/usr/lib"  -O3 -o obj/sha256.o sha256.cpp
sha256.cpp:12:23: error: xmmintrin.h: No such file or directory
sha256.cpp:40: error: '__m128i' does not name a type
sha256.cpp:44: error: '__m128i' does not name a type
sha256.cpp:48: error: '__m128i' does not name a type
sha256.cpp:52: error: '__m128i' does not name a type
sha256.cpp:62: error: expected ',' or '...' before 'x'
sha256.cpp:62: error: ISO C++ forbids declaration of '__m128i' with no type
sha256.cpp: In function 'unsigned int store32(int)':
sha256.cpp:63: error: '__m128i' does not name a type
sha256.cpp:64: error: 'union store32(int)::<anonymous>' has no member named 'x'
sha256.cpp:64: error: 'x' was not declared in this scope
sha256.cpp:65: error: 'i' was not declared in this scope
sha256.cpp: At global scope:
sha256.cpp:68: error: expected ',' or '...' before 'x'
sha256.cpp:68: error: ISO C++ forbids declaration of '__m128i' with no type
sha256.cpp: In function 'void store_epi32(int)':
sha256.cpp:69: error: '__m128i' does not name a type
sha256.cpp:70: error: 'union store_epi32(int)::<anonymous>' has no member named 'x'
sha256.cpp:70: error: 'x' was not declared in this scope
sha256.cpp:71: error: 'x0' was not declared in this scope
sha256.cpp:71: error: 'x1' was not declared in this scope
sha256.cpp:71: error: 'x2' was not declared in this scope
sha256.cpp:71: error: 'x3' was not declared in this scope
sha256.cpp: At global scope:
sha256.cpp:82: error: variable or field 'dumpreg' declared void
sha256.cpp:82: error: '__m128i' was not declared in this scope
sha256.cpp:82: error: expected primary-expression before 'char'
gmake: *** [obj/sha256.o] Error 1


twobits?

[twobits]

# gmake -f Makefile2 bitcoind
g++ -c -O2 -fstack-protector -Wno-invalid-offsetof -Wformat -g -D__WXDEBUG__ -D__WXGTK__ -DNOPCH -DFOURWAYSSE2 -DUSE_SSL -D__BSD__ -I"/home/root2/bitcoind/deps/include" -I"/usr/local/lib/wx/include/gtk2-unicode-release-2.9" -I"/usr/local/include/db47" -I"/usr/local/include" -L"/usr/local/lib/db47" -L"/home/root2/bitcoind/deps/lib" -L"/usr/lib"  -O3 -o obj/sha256.o sha256.cpp

twobits?

I would say the next step is to take out the -DFOURWAYSSE2   since the sparc is not going to have x86 SSE2 instructions.

[sn-2011a]

Thanks again twobits for your reply to my last post. I got past that hurdle and i fell like im almost there.

I have finally made it to the linking stage. Now i have a problem with ld. It tells me it cant find the first boost library in its list, boost_system. 

Code:

# gmake -f Makefile2 bitcoind
g++ -O2 -fstack-protector -Wno-invalid-offsetof -Wformat -g -D__WXDEBUG__ -D__WXGTK__ -DNOPCH -DUSE_SSL -D__BSD__ -I"/home/root2/bitcoind/deps/include" -I"/usr/local/lib/wx/include/gtk2-unicode-release-2.9" -I"/usr/local/include/db47" -I"/usr/local/include" -L"/usr/local/lib/db47" -L"/home/root2/bitcoind/deps/lib" -L"/usr/lib" -o bitcoind obj/nogui/util.o obj/nogui/script.o obj/nogui/db.o obj/nogui/net.o obj/nogui/irc.o obj/nogui/main.o obj/nogui/rpc.o obj/nogui/init.o cryptopp/obj/sha.o cryptopp/obj/cpu.o obj/sha256.o -dead-strip -l boost_system-mt -l boost_filesystem-mt -l boost_program_options-mt -l boost_thread-mt -l db_cxx -l ssl -l crypto -Wl,-Bdynamic -l z -l pthread
/usr/bin/ld: cannot find -lboost_system-mt
collect2: ld returned 1 exit status
gmake: *** [bitcoind] Error 1

Proof it is there:

Code:

# find /home/root2/bitcoind/deps/ | grep boost_system
/home/root2/bitcoind/deps/boost_1_37_0/bin.v2/libs/system/build/gcc-4.2.1/release/threading-multi/libboost_system-gcc42-mt-1_37.so.1.37.0
/home/root2/bitcoind/deps/boost_1_37_0/bin.v2/libs/system/build/gcc-4.2.1/release/link-static/threading-multi/libboost_system-gcc42-mt-1_37.a
/home/root2/bitcoind/deps/lib/libboost_system-gcc42-mt-1_37.so.1.37.0
/home/root2/bitcoind/deps/lib/libboost_system-gcc42-mt-1_37.so
/home/root2/bitcoind/deps/lib/libboost_system-gcc42-mt.so
/home/root2/bitcoind/deps/lib/libboost_system-gcc42-mt-1_37.a
/home/root2/bitcoind/deps/lib/libboost_system-gcc42-mt.a

I modified the g++ command to include the absolute path to this lib and ld still "cant find" it.

Code:

g++ -O2 -fstack-protector -Wno-invalid-offsetof -Wformat -g -D__WXDEBUG__ -D__WXGTK__ -DNOPCH -DUSE_SSL -D__BSD__ -I"/home/root2/bitcoind/deps/include" -L"/home/root2/bitcoind/deps/lib" -o bitcoind obj/nogui/util.o obj/nogui/script.o obj/nogui/db.o obj/nogui/net.o obj/nogui/irc.o obj/nogui/main.o obj/nogui/rpc.o obj/nogui/init.o cryptopp/obj/sha.o cryptopp/obj/cpu.o obj/sha256.o -dead-strip -l /home/root2/bitcoind/deps/lib/libboost_system-gcc42-mt-1_37.a -l boost_filesystem-mt -l boost_program_options-mt -l boost_thread-mt -l db_cxx -l ssl -l crypto -Wl,-Bdynamic -l z -l pthread

/usr/bin/ld: cannot find -l/home/root2/bitcoind/deps/lib/libboost_system-gcc42-mt-1_37.a
collect2: ld returned 1 exit status

Any ideas?

[twobits]

Maybe take off the extenstion, and/or add a -Bstatic flag before those libraries.
 

[Venkatesh Srinivas]

Bitcoind is unlikely to work on your OpenBSD sparc64 system; the client does not run on big-endian systems.

[sn-2011a]

I jumped the last hurdle by changing all the boost_xxx-mt to boost_xxx-gcc42-mt.

Now i am getting errors from the boost libs:

Code:

# gmake -f Makefile2 bitcoind
g++ -c -O2 -fstack-protector -Wno-invalid-offsetof -Wformat -g -D__WXDEBUG__ -D__WXGTK__ -DNOPCH -DUSE_SSL -D__BSD__ -I"/home/root2/bitcoind/deps/include" -I"/home/root2/bitcoind/deps/include/boost-1_37" -L"/home/root2/bitcoind/deps/lib"  -o obj/nogui/rpc.o rpc.cpp
In file included from /home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/context_service.hpp:30,
                 from /home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/context.hpp:22,
                 from /home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl.hpp:19,
                 from rpc.cpp:12:
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp: In member function 'void boost::asio::ssl::detail::openssl_context_service::create(SSL_CTX*&, boost::asio::ssl::context_base::method)':
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:74: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:77: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:80: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:83: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:86: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:89: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:92: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:95: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:98: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:101: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:104: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
/home/root2/bitcoind/deps/include/boost-1_37/boost/asio/ssl/detail/openssl_context_service.hpp:107: error: invalid conversion from 'const SSL_METHOD*' to 'SSL_METHOD*'
gmake: *** [obj/nogui/rpc.o] Error 1
#                                                                                                              

Here is a snip from the offending file

Code:

  // Create a new context implementation.
  void create(impl_type& impl, context_base::method m)
  {
    ::SSL_METHOD* ssl_method = 0;
    switch (m)
    {
    case context_base::sslv2:
      ssl_method = ::SSLv2_method();
      break;
    case context_base::sslv2_client:
      ssl_method = ::SSLv2_client_method();
      break;
    case context_base::sslv2_server:
      ssl_method = ::SSLv2_server_method();
      break;
    case context_base::sslv3:
      ssl_method = ::SSLv3_method();
      break;
    case context_base::sslv3_client:
      ssl_method = ::SSLv3_client_method();
      break;

Any C++ help greatly appreciated.

[sn-2011a]

Nevermind,
I got it.

I changed:

Code:

  // Create a new context implementation.
  void create(impl_type& impl, context_base::method m)
  {
    ::SSL_METHOD* ssl_method = 0;
    switch (m)
    {
    case context_base::sslv2:
      ssl_method = ::SSLv2_method();
      break;
    case context_base::sslv2_client:
      ssl_method = ::SSLv2_client_method();
      break;

to

Code:

 
 // Create a new context implementation.
  void create(impl_type& impl, context_base::method m)
  {
    const ::SSL_METHOD* ssl_method = 0;
    switch (m)
    {
    case context_base::sslv2:
      ssl_method = ::SSLv2_method();
      break;
    case context_base::sslv2_client:
      ssl_method = ::SSLv2_client_method();
      break;

Now I've got bitcoind compiled and running on OpenBSD 4.9 Sparc64. What do i do next?

[jh1523]

Now I've got bitcoind compiled and running on OpenBSD 4.9 Sparc64. What do i do next?

Write a detailed howto for other people to use.

[Venkatesh Srinivas]

Start it and try it; but I suspect it will fail early.

On another big-endian system, it trips this assert: assert(block.hashMerkleRoot == uint256("0x4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b")); fairly quickly.

[sn-2011a]

Venkatesh Srinivas, it works over here. Did you take out the SSE2 stuff?

Here is the HOWTO:

Compiling Bitcoind on OpenBSD4.9 Sparc64

This doc will guide you to compiling Bitcoind on OpenBSD-4.9 Sparc64

---Getting Ready---
Besides Bitcoind's direct dependncys you will need to install gmake and wget from the packages collection
Bitconind's dependencys are not satisfied with the packages or ports collection of OpenBSD4.9.
http://mirror.ece.vt.edu/pub/OpenBSD/4.9/packages/sparc64/  <-packages
Berkley DB 4.6.21p4 is there but we need 4.7
Boost 1.42 is there but we need 1.37
OpenSSL is unkonwn
Some of these deps are a couple of years old, but we will get them anyways.
We will need: Boost 1.37, Berkley DB 4.7.25, openSSL 0.9.8g, and Bitcoind R251
Make a directory for this compilation project and the dependencys.
Lets get all the downloading out of the way first

Code:

cd ~
mkdir bitcoind
mkdir bitcoind/deps
cd bitcoind/deps
wget "http://www.openssl.org/source/openssl-0.9.8g.tar.gz"
tar -xzf openssl-0.9.8g.tar.gz
wget "http://download.oracle.com/berkeley-db/db-4.7.25.tar.gz"
tar -xzf db-4.7.25.tar.gz

Go online to http://sourceforge.net/projects/boost/files/boost/1.37.0/
and download and and unzip it like the others

Now we will use svn to get Bitcoin. SVN will make a folder called trunk to put the code in.

Code:

cd ~/bitcoind/
svn checkout https://bitcoin.svn.sourceforge.net/svnroot/bitcoin/trunk

--Compiling and building dependencys--

Lets start with Boost

Code:

cd ~/bitcoind/deps/boost_1_37_0
./configure --prefix=~/bitcoind/deps link=static runtime-link=static install
make

Now Berkley DB

Code:

cd ~/bitcoind/deps/db-4.7.25
cd build_unix
../dist/configure --prefix=~/bitcoind/deps --enable-cxx
make
make install

Finally we need need to finess openssl a bit. DES was giving me problems so i excluded it, you might not have to. I also had to change a file because of use of a depreciate option to the m4 macro processor.

Code:

cd ~/bitcoind/deps/openssl-0.9.8g
#you may wish to change the line below if you are using a diffrent arch. or want to try to include des
./Configure --prefix=~/bitcoind/deps --openssldir=~/bitcoind/deps/openssl no-des BSD-sparc64
make depend
#if you get an error running the command below you might need to change "m4 -B" to "m4 -D" because -B is depreciated
make

---OpenBSD specific changes to the code of Bitcoin ---
net.cpp and irc.cpp need some changes to work on OpenBSD
OpenBSD does not have MSG_NOSIGNAL so we will "ignore it" by adding these three lines below the headers in each of the two files

Code:

#ifndef MSG_NOSIGNAL
#define MSG_NOSIGNAL 0
#endif 

now irc.cpp is done but net.cpp need some more work

Code:

vi ~/bitcoind/trunk/net.cpp

use the / key to search for SO_NOSIGPIPE. This is not supported in OpenBSD either so we will take it out. We need to yank 4 lines from #ifdefBSD down to #endif
you will find one more of these by searching again for the same thing, take it out

Finally we will drop in our own makefile and be ready to compile bitcoind

Code:

cd ~/bitcoind/trunk/
cat > Makefile.OpenBSD
#Bitcoind makefile for OpenBSD 
INCLUDEPATHS= \
 -I"~/bitcoind/deps/include" \
 -I"~/bitcoind/deps/include/boost-1_37"

LIBPATHS= \
 -L"~/bitcoind/deps/lib"

WXLIBS= \
 -Wl,-Bstatic \
   -l wx_gtk2ud-2.9 \
 -Wl,-Bdynamic \
   -l gtk-x11-2.0 \
   -l SM

LIBS= -dead-strip \
   -l boost_system-gcc42-mt \
   -l boost_filesystem-gcc42-mt \
   -l boost_program_options-gcc42-mt \
   -l boost_thread-gcc42-mt \
   -l db_cxx \
   -l ssl \
   -l crypto \
 -Wl,-Bdynamic \
   -l z \
   -l pthread

DEFS=-D__WXGTK__ -DNOPCH -DUSE_SSL -D__BSD__
DEBUGFLAGS=-g -D__WXDEBUG__
CFLAGS=-O2 -fstack-protector -Wno-invalid-offsetof -Wformat $(DEBUGFLAGS) $(DEFS) $(INCLUDEPATHS) $(LIBPATHS)
HEADERS=headers.h strlcpy.h serialize.h uint256.h util.h key.h bignum.h base58.h \
    script.h db.h net.h irc.h main.h rpc.h uibase.h ui.h noui.h init.h

OBJS= \
    obj/util.o \
    obj/script.o \
    obj/db.o \
    obj/net.o \
    obj/irc.o \
    obj/main.o \
    obj/rpc.o \
    obj/init.o \
    cryptopp/obj/sha.o \
    cryptopp/obj/cpu.o


all: bitcoin


obj/%.o: %.cpp $(HEADERS)
        g++ -c $(CFLAGS) -DGUI -o $@ $<

cryptopp/obj/%.o: cryptopp/%.cpp
        g++ -c $(CFLAGS) -O3 -o $@ $<

obj/sha256.o: sha256.cpp
        g++ -c $(CFLAGS)  -O3 -o $@ $<

bitcoin: $(OBJS) obj/ui.o obj/uibase.o obj/sha256.o
        g++ $(CFLAGS) -o $@ $^ $(WXLIBS) $(LIBS)


obj/nogui/%.o: %.cpp $(HEADERS)
        g++ -c $(CFLAGS) -o $@ $<

bitcoind: $(OBJS:obj/%=obj/nogui/%) obj/sha256.o
        g++ $(CFLAGS) -o $@ $^ $(LIBS)


clean:
        -rm -f obj/*.o
        -rm -f obj/nogui/*.o
        -rm -f cryptopp/obj/*.o
        -rm -f headers.h.gch

---Compiling Bitcoind---

Code:

cd ~/bitcoind/trunk/
gmake -f Makefile.OpenBSD bitcoind

Enjoy

[Venkatesh Srinivas]

Works for me on OpenBSD/i386.

The SO_NOSIGPIPE fix is in bitcoin's git master; I submitted it to deal with DragonFly BSD. I've submitted a patch for the MSG_NOSIGNAL definition as well to the list.

But I'm confused as to how it is working on Sparc64; isn't sparc64 a big-endian machine? The C++ bitcoin client isn't supposed to run on BE machines... have you been able to download the blockchain and everything?

Thanks,
-- vs

[zer0]

Bumping this because this guy's method still works for OpenBSD 5.0 + latest bitcoind source
Probably should put it in the bitcoin wiki or possibly on the main bitcoin.org page

Contacted some pkg maintainers they may throw it into the cvs tree after some auditing. Great success
Its already in the ports tree