Detailed information about datadir

[DeathAndTaxes]

Looking for detailed information on the data directory and file formats for QT client (0.8.3).

There is this wiki page but it is pretty sparse:
https://en.bitcoin.it/wiki/Data_directory

For example what format is the peers.dat file in?

The chainstate sub folder contains leveldb representation of the UXTO right?
What is the significance of multiple xxxxxx.sst files?
There is also LOCK, LOG, and MANIFEST-xxxxxx files?

The "blocks" subfolder contains leveldb representation of raw blocks right?
Why are the files of difference size?

Code:

08/05/2012  02:16 AM     2,097,361,271 blk00000.dat
12/16/2012  02:37 PM     2,097,361,271 blk00001.dat
12/16/2012  07:04 PM     2,097,299,522 blk00002.dat
02/22/2013  01:23 PM     1,879,601,111 blk00003.dat
02/27/2013  07:07 AM       134,217,728 blk00004.dat
03/04/2013  02:05 AM       134,217,728 blk00005.dat
03/08/2013  01:52 PM       134,217,728 blk00006.dat
03/13/2013  05:51 AM       134,217,728 blk00007.dat
03/18/2013  11:53 PM       134,217,728 blk00008.dat
03/24/2013  03:29 AM       134,217,728 blk00009.dat
03/29/2013  04:37 AM       134,217,728 blk00010.dat
04/02/2013  06:30 PM       134,217,728 blk00011.dat
04/07/2013  01:42 AM       134,217,728 blk00012.dat
04/11/2013  06:15 AM       134,217,728 blk00013.dat
04/16/2013  12:28 AM       134,217,728 blk00014.dat
04/21/2013  04:08 AM       134,217,728 blk00015.dat
04/25/2013  09:57 PM       134,217,728 blk00016.dat
05/01/2013  09:47 AM       134,217,728 blk00017.dat
05/06/2013  02:32 PM       134,217,728 blk00018.dat
05/11/2013  12:56 AM       134,217,728 blk00019.dat
05/15/2013  07:33 AM       134,217,728 blk00020.dat
05/20/2013  09:52 AM       134,217,728 blk00021.dat
05/25/2013  11:00 AM       134,217,728 blk00022.dat
05/30/2013  03:57 PM       134,217,728 blk00023.dat
06/04/2013  05:22 PM       134,217,728 blk00024.dat
06/12/2013  11:01 AM       134,217,728 blk00025.dat
06/16/2013  06:07 PM       134,035,313 blk00026.dat
07/02/2013  06:03 PM       134,216,638 blk00027.dat
07/02/2013  06:16 PM       134,108,901 blk00028.dat
07/03/2013  06:25 PM        83,886,080 blk00029.dat
02/22/2013  01:42 PM       264,241,152 rev00000.dat
02/22/2013  01:54 PM       242,221,056 rev00002.dat
02/22/2013  02:25 PM       236,978,176 rev00003.dat
02/27/2013  07:07 AM        17,825,792 rev00004.dat
03/04/2013  02:05 AM        16,777,216 rev00005.dat
03/08/2013  01:52 PM        17,825,792 rev00006.dat
03/13/2013  05:51 AM        16,777,216 rev00007.dat
03/18/2013  11:53 PM        16,777,216 rev00008.dat
03/24/2013  03:29 AM        16,777,216 rev00009.dat
03/29/2013  04:37 AM        16,777,216 rev00010.dat
04/02/2013  06:30 PM        17,825,792 rev00011.dat
04/07/2013  01:42 AM        16,777,216 rev00012.dat
04/11/2013  06:15 AM        16,777,216 rev00013.dat
04/16/2013  12:28 AM        16,777,216 rev00014.dat
04/21/2013  04:08 AM        16,777,216 rev00015.dat
04/25/2013  09:57 PM        16,777,216 rev00016.dat
05/01/2013  09:47 AM        16,777,216 rev00017.dat
05/06/2013  02:32 PM        16,777,216 rev00018.dat
05/11/2013  12:56 AM        16,777,216 rev00019.dat
05/15/2013  07:33 AM        16,777,216 rev00020.dat
05/20/2013  09:52 AM        16,777,216 rev00021.dat
05/25/2013  11:00 AM        16,777,216 rev00022.dat
05/30/2013  03:57 PM        17,825,792 rev00023.dat
06/04/2013  05:22 PM        17,825,792 rev00024.dat
06/12/2013  11:01 AM        17,825,792 rev00025.dat
06/16/2013  06:07 PM        17,127,799 rev00026.dat
07/02/2013  06:03 PM        17,186,353 rev00027.dat
07/02/2013  06:16 PM        17,151,788 rev00028.dat
07/03/2013  06:25 PM         9,437,184 rev00029.dat

For example there are 30 blkxxxxxx files the first few are very large then most are 134MB.  The last one is smaller but that would make sense as it is the block file in progress.
Why 134MB?  Why roughly 2GB for the first two block files and then the third one breaks that pattern?

[1714032440]

1714032440

[1714032440]

1714032440

[1714032440]

1714032440

[etotheipi]

For example there are 30 blkxxxxxx files the first few are very large then most are 134MB.  The last one is smaller but that would make sense as it is the block file in progress.
Why 134MB?  Why roughly 2GB for the first two block files and then the third one breaks that pattern?

If you redownload the blockchain, they will be 134 MB.  But if your blockfiles were mostly downloaded prior to version 0.8, then it used a 2 GB blkfile size, and just kept them when they switched it in 0.8.

If you were to wipe the directory and redownload with 0.8+, it would be about 70 files of 134 MB each.  If you redownloaded with version 0.7 or prior, it would be 5 files of 2 GB each.

[DeathAndTaxes]

Thanks.  That makes sense and yes this is from a very old install so it fits.
 
Any idea on the format of peers.dat file?

[DeathAndTaxes]

Found a little more info here ....

Used in 0.8.0:
* wallet.dat: personal wallet (BDB) with keys and transactions
* peers.dat: peer IP address database (custom format); since 0.7.0
* blocks/blk000??.dat: block data (custom, 128 MiB per file); since 0.8.0
* blocks/rev000??.dat; block undo data (custom); since 0.8.0 (format changed since pre-0.
* blocks/index/*; block index (LevelDB); since 0.8.0
* chainstate/*; block chain state database (LevelDB); since 0.8.0

https://github.com/bitcoin/bitcoin/blob/cb2e1bdaa902197ba0f04aebd00420a4680548e0/doc/files.txt

ok "custom format".  Well that helps.

[jgarzik]

The chainstate sub folder contains leveldb representation of the UXTO right?

Yes.

What is the significance of multiple xxxxxx.sst files?
There is also LOCK, LOG, and MANIFEST-xxxxxx files?

These are leveldb internal storage setup.  Inherent in leveldb design.  Analogy: Berkeley DB's low level storage format.

The "blocks" subfolder contains leveldb representation of raw blocks right?

No.  This is the raw bitcoin-specific wire format of blocks.

Why are the files of difference size?

Bitcoin blocks differ in size.

Also, some blk*.dat are larger because they were originally that way when imported the first time.

If you use bootstrap.dat to initialize a new database then all your files should be nearly the same size.