fluffypony
Donator
Legendary
Offline
Activity: 1274
Merit: 1060
GetMonero.org / MyMonero.com
|
|
May 06, 2015, 03:19:35 PM |
|
ok i managed to convert on windows 7 x64 with batch size 1000 using blockchain_converter. was really fast(maybe 30min), but: - data.mdb uses now more than 16gb, this is not normal no? this looks like some crazy pre allocation. same happens when exporting / importing. - i want to highlight again, on windows monero uses now %programData% as default location instead of %appData% like in 0.8.8.6. i am fully synced right now and bitmonerod uses 31 Megabyte of ram good job devs !!! yeah, thats the size of my data.mdb. Fluffypony said it something with windows not liking sparse files. Whatever that means. weird - I think my data.mdb was in %appData%/bitmonerod/LMDB , not %programData%, but I'll double check when I get home. So basically what happens is we let LMDB pre-allocate 16gb for the blockchain, and then we'll add in a function to automatically pre-allocate more as needed. This is great on every OS that has sparse file support, as the size on disk will only be like 5gb (ie. the bit with actual data). However, it appears that on some flavours of Windows, or some Windows environments, that it takes up the full 16gb on disk. We're currently working on a more efficient pre-allocation system that will preallocate an additional 1gb once there is < 500mb preallocation available, as that should be fast enough for the foreseeable future whilst solving the pre-allocation Windows issue.
|
|
|
|
Shrikez
|
|
May 06, 2015, 03:36:23 PM |
|
Just to add my stats for the DB build on Linux Mint:
size of data.mdb is currently 6,071 GB bitmonerod memory use fully synched: 58 MB
|
Die Würde des Menschen ist unantastbar
|
|
|
primer-
Legendary
Offline
Activity: 1092
Merit: 1000
|
|
May 06, 2015, 04:58:52 PM |
|
LMDB version not official. Blockchain conversion tools broken. No official GUI. No TAILS binaries available Freebazaar nowhere near complete, last git submission 28 days ago. DwarfPool scam ongoing, efficiency below 80%
I can't wait to see fluffyponny in person later this month, got a full bag of rotten eggs and tomatoes just for him..
|
|
|
|
equipoise
|
|
May 06, 2015, 05:17:55 PM |
|
Win 7 64x, uses ~49MB RAM, pre-allocated blockchain space on HDD 16GB. Works stable for now. 'q' doesn't work for 'exit' like in the previous versions. It'll be another 24/7 node.
|
|
|
|
infovortice2013
Legendary
Offline
Activity: 1176
Merit: 1000
|
|
May 06, 2015, 05:23:21 PM |
|
it new wallet with nice gui ando secure working available?
|
|
|
|
oblox
Legendary
Offline
Activity: 1442
Merit: 1018
|
|
May 06, 2015, 06:04:05 PM |
|
it new wallet with nice gui ando secure working available?
Engrish?
|
|
|
|
bigj
|
|
May 06, 2015, 06:04:54 PM |
|
LMDB version not official. Blockchain conversion tools broken. No official GUI. No TAILS binaries available Freebazaar nowhere near complete, last git submission 28 days ago. DwarfPool scam ongoing, efficiency below 80%
I can't wait to see fluffyponny in person later this month, got a full bag of rotten eggs and tomatoes just for him..
Since you are obviously a fan of Monero, why don't you get your hands dirty and contribute?
|
|
|
|
|
kazuki49
|
|
May 06, 2015, 07:36:52 PM |
|
LMDB version not official. Blockchain conversion tools broken. No official GUI. No TAILS binaries available Freebazaar nowhere near complete, last git submission 28 days ago. DwarfPool scam ongoing, efficiency below 80%
I can't wait to see fluffyponny in person later this month, got a full bag of rotten eggs and tomatoes just for him..
Since you are obviously a fan of Monero, why don't you get your hands dirty and contribute? I try to see the good side of comments, at least we have a lot of people who cares about Monero
|
|
|
|
mmortal03
Legendary
Offline
Activity: 1762
Merit: 1011
|
|
May 06, 2015, 07:42:37 PM |
|
ok i managed to convert on windows 7 x64 with batch size 1000 using blockchain_converter. was really fast(maybe 30min), but: - data.mdb uses now more than 16gb, this is not normal no? this looks like some crazy pre allocation. same happens when exporting / importing. - i want to highlight again, on windows monero uses now %programData% as default location instead of %appData% like in 0.8.8.6. i am fully synced right now and bitmonerod uses 31 Megabyte of ram good job devs !!! yeah, thats the size of my data.mdb. Fluffypony said it something with windows not liking sparse files. Whatever that means. weird - I think my data.mdb was in %appData%/bitmonerod/LMDB , not %programData%, but I'll double check when I get home. It was %programdata% for me, too. Btw, GingerAle, I've discovered a problem with running the latest binaries with MoneroX: the bitmonerod process doesn't close when exiting MoneroX. I see in the log that it *does* receive a stop signal from MoneroX, and p2p net loop does eventually stop, but the daemon process doesn't close. I assumed that MoneroX just hasn't been updated to use the current RPC commands to stop the daemon, but... I tried running the stop_daemon command myself from the command line, outside of MoneroX, but that also doesn't seem to work. It works just fine if I've run bitmonerod myself, so I know I'm doing it right. I can't tell if bitmonerod after closing MoneroX is just stuck in some buggy state, or if there is some sort of process communication issue that isn't obvious to me. It doesn't seem to be a permissions issue, as the process shows up as loaded by my username in Windows in both instances. I can still get the local block height while it is in this state, and the daemon state *is* detected when running stop_daemon, exit, and fast_exit from the command line in various combinations, so I know that the communications are being received by it to some extent... the process just doesn't close.
|
|
|
|
GingerAle
Legendary
Offline
Activity: 1260
Merit: 1008
|
|
May 06, 2015, 09:19:14 PM |
|
ok i managed to convert on windows 7 x64 with batch size 1000 using blockchain_converter. was really fast(maybe 30min), but: - data.mdb uses now more than 16gb, this is not normal no? this looks like some crazy pre allocation. same happens when exporting / importing. - i want to highlight again, on windows monero uses now %programData% as default location instead of %appData% like in 0.8.8.6. i am fully synced right now and bitmonerod uses 31 Megabyte of ram good job devs !!! yeah, thats the size of my data.mdb. Fluffypony said it something with windows not liking sparse files. Whatever that means. weird - I think my data.mdb was in %appData%/bitmonerod/LMDB , not %programData%, but I'll double check when I get home. It was %programdata% for me, too. Btw, GingerAle, I've discovered a problem with running the latest binaries with MoneroX: the bitmonerod process doesn't close when exiting MoneroX. I see in the log that it *does* receive a stop signal from MoneroX, and p2p net loop does eventually stop, but the daemon process doesn't close. I assumed that MoneroX just hasn't been updated to use the current RPC commands to stop the daemon, but... I tried running the stop_daemon command myself from the command line, outside of MoneroX, but that also doesn't seem to work. It works just fine if I've run bitmonerod myself, so I know I'm doing it right. I can't tell if bitmonerod after closing MoneroX is just stuck in some buggy state, or if there is some sort of process communication issue that isn't obvious to me. It doesn't seem to be a permissions issue, as the process shows up as loaded by my username in Windows in both instances. I can still get the local block height while it is in this state, and the daemon state *is* detected when running stop_daemon, exit, and fast_exit from the command line in various combinations, so I know that the communications are being received by it to some extent... the process just doesn't close. Yeah, i don't know if this is specific to plopping the new bins into MoneroX or not .... the "bitmonerod ignores stop signal" thing has been present for a while - it happens on my linux box, it happens on my windows box. I have noticed though that this state occurs more frequently if I try to stop bitmonerod soon after starting it... so if its running for a while, it will exit clean. If its running for < 1 minute, it goes into the ignore state. Just confirmed, my data files are in %AppData%. I'm on windows 7.
|
|
|
|
|
mmortal03
Legendary
Offline
Activity: 1762
Merit: 1011
|
|
May 06, 2015, 10:48:38 PM |
|
Yeah, i don't know if this is specific to plopping the new bins into MoneroX or not .... the "bitmonerod ignores stop signal" thing has been present for a while - it happens on my linux box, it happens on my windows box. I have noticed though that this state occurs more frequently if I try to stop bitmonerod soon after starting it... so if its running for a while, it will exit clean. If its running for < 1 minute, it goes into the ignore state.
Yeah, I'm totally aware of that, but this seems to be a special case. I'll run MoneroX for a few hours, close it, wait for p2p net loop to stop in the log, and bitmonerod still doesn't close. This seems to be reproducible. Just confirmed, my data files are in %AppData%. I'm on windows 7. Where is it creating your bitmonero.log? It's putting mine under %ProgramData%, on Windows 7.
|
|
|
|
GingerAle
Legendary
Offline
Activity: 1260
Merit: 1008
|
|
May 06, 2015, 10:58:23 PM |
|
Yeah, i don't know if this is specific to plopping the new bins into MoneroX or not .... the "bitmonerod ignores stop signal" thing has been present for a while - it happens on my linux box, it happens on my windows box. I have noticed though that this state occurs more frequently if I try to stop bitmonerod soon after starting it... so if its running for a while, it will exit clean. If its running for < 1 minute, it goes into the ignore state.
Yeah, I'm totally aware of that, but this seems to be a special case. I'll run MoneroX for a few hours, close it, wait for p2p net loop to stop in the log, and bitmonerod still doesn't close. This seems to be reproducible. Just confirmed, my data files are in %AppData%. I'm on windows 7. Where is it creating your bitmonero.log? It's putting mine under %ProgramData%, on Windows 7. %AppData% I wonder if I built a slightly older version.
|
|
|
|
btell
Member
Offline
Activity: 119
Merit: 10
|
|
May 06, 2015, 11:46:02 PM |
|
LMDB version not official. Blockchain conversion tools broken. No official GUI. No TAILS binaries available Freebazaar nowhere near complete, last git submission 28 days ago. DwarfPool scam ongoing, efficiency below 80%
I can't wait to see fluffyponny in person later this month, got a full bag of rotten eggs and tomatoes just for him..
Agree, fluffypony moves like a true pony. Load him by heavy bag of dollars. Let pony to work I see, many users report successful blockchain conversion to LMDB with batch size = 1000. The little issue is 16GB sparse database file. 'Sparceness' is well known feature to programmers, it's normal. Actual database file size is much less, say 3-5 Gb. Note memory fingerprint instead - users do report 30-60Mb. IT IS THE WIN.
|
|
|
|
othe
|
|
May 07, 2015, 12:00:40 AM |
|
LMDB version not official. Blockchain conversion tools broken. No official GUI. No TAILS binaries available Freebazaar nowhere near complete, last git submission 28 days ago. DwarfPool scam ongoing, efficiency below 80%
I can't wait to see fluffyponny in person later this month, got a full bag of rotten eggs and tomatoes just for him..
awesome, i´ll also be there, hope u can run faster than i can throw a beer bottle.
|
|
|
|
|
smooth
Legendary
Offline
Activity: 2968
Merit: 1198
|
|
May 07, 2015, 12:55:03 AM |
|
LMDB version not official. Blockchain conversion tools broken. No official GUI. No TAILS binaries available Freebazaar nowhere near complete, last git submission 28 days ago. DwarfPool scam ongoing, efficiency below 80%
I can't wait to see fluffyponny in person later this month, got a full bag of rotten eggs and tomatoes just for him..
Agree, fluffypony moves like a true pony. Load him by heavy bag of dollars. Let pony to work I see, many users report successful blockchain conversion to LMDB with batch size = 1000. The little issue is 16GB sparse database file. 'Sparceness' is well known feature to programmers, it's normal. Actual database file size is much less, say 3-5 Gb. Note memory fingerprint instead - users do report 30-60Mb. IT IS THE WIN. Problem is sparse either doesn't work on Windows (or Mac, I think), or doesn't work well, at least in some configurations. Nevertheless this will be addressed once the ability to expand the database is added. Then the starting size can be backed off., at least for non-sparse deployments.
|
|
|
|
Brilliantrocket
|
|
May 07, 2015, 01:03:44 AM |
|
I see that one of the development goals is the implementation of BerkleyDB as a fallback. Will this be as time intensive as the work on LMDB has been?
|
|
|
|
othe
|
|
May 07, 2015, 01:07:57 AM |
|
I see that one of the development goals is the implementation of BerkleyDB as a fallback. Will this be as time intensive as the work on LMDB has been?
It´s already implemented and works on Raspberry PI etc. There are currently 3 database types with the DB interface: - LMDB (normal pcs) - Berkley DB (embedded devices like RPi) - In Memory dB (not the old one, it also uses the DB interface, for whatever services who need the whole db in ram for fast searching etc.)
|
|
|
|
|