Ask Ninja: Backup up PMs

[Vod]

Hey LoyceV, sorry to put you on the spot, but I think we could open up a new forum subseries called "Ask LoyceV".  

Would it be possible for you to write a quick script that could back up our private messages?  You mentioned once before it wouldn't be too difficult, and I know there is a lot of valuable data people want to keep in those messages.
https://bitcointalk.org/index.php?topic=5277566.msg55265547#msg55265547

(Edit:  Ninja provided the code)

BATTLE OF THE CODERS

[1714248501]

1714248501

[1714248501]

1714248501

[1714248501]

1714248501

[1714248501]

1714248501

[1714248501]

1714248501

[tranthidung]

Ask Vod first  

You did coded it and published it.
Do you want a utility to save all your PMs?

Your product

• Backup/Sort Private Messages/PM/DM
• https://github.com/martin2025/PPM

Did your forget about your own product or just want to find something with extra features?

[TryNinja]

Here's a quick one I just made.

https://gist.github.com/ninjastic/bc568895778132e1414564147d3a9aaf

1. Open the the forum on any page.
2. Open your browser's developer tools.
3. Go to the Console tab.
4. Paste and run the code. Do not close the tab and avoid using the forum in the meanwhile so you don't get rate limited.

After finishing it, a file called pm.json will be downloaded with the format:

Code:

[
  {
    "type": "inbox",
    "title": "Here are my coins",
    "date": "2010-03-22, 12:33:53",
    "to": "TryNinja",
    "toUrl": "https://bitcointalk.org/index.php?action=profile;u=557798",
    "responded": true,
    "message": "Hello TryNinja,<br><br>I will be sending my coins to you tomorrow, thank you <img src=\"https://bitcointalk.org/Smileys/default/smiley.gif\" alt=\"Smiley\" border=\"0\"><br><br>Best,<br>satoshi",
    "author": "satoshi",
    "authorUrl": "https://bitcointalk.org/index.php?action=profile;u=3"
  }
]

Didn't hard test it, so idk... Worked for my inbox/outbox.

[Vod]

Ask Vod first  

You did coded it and published it.
Do you want a utility to save all your PMs?

No, if you follow the adventures in that thread, you'll read my one test subject could not get it working.   :/

I was also coding in windows, and people here don't want to run an executable.

Thanks Ninja!  You beat LoyceV to it (probably due to where the sun was) so I will change the title.

Edit:  LoyceV has a different PM tool:  https://bitcointalk.org/index.php?topic=5386000.

[LoyceV]

Ninja's don't need sleep, I do

I already have my PM Publisher, and I can think of something else. But I don't think anyone would want it, since I'd need your password.

Here it goes (hypothetical):

• Make sure you have staked a Bitcoin address
• Change your password into something very long
• PM my to-be-created-bot (running on a newly created and not publicly known server) your password and captcha bypass code
• Give my bot some time to save all PMs, one file per PM
• I'll create and publish SHA-256 checksums for each PM.
• My bot creates a compressed archive with all files, encrypted with the password you provided
• My bot deletes all your PMs from my server
• Change your password again so I don't know it
• My bot deletes the encrypted archive a day later (note that my bot won't delete your PM, my bot can only download, not edit)
• Archive the SHA-256 checksums

Pros:

• You have a backup of your PMs
• You can prove authenticity of any PM, even after deleting it from your inbox, to anyone who trusts me (and my bot). You'll have a file, I'll have a checksum. The checksum doesn't even need to say who it's from, that will be in the file.

Cons:

• You have to trust me (keeping data private, not changing your password, not snooping around, etc.)
• It doesn't prove anything to anyone who doesn't trust me
• You'll have to trust my (cheap) server and it's hosting
• A big con: myips.php gives away my server IP to any user of this service, which increases the risk of point 3 (although a VPN on the server could solve this)
• (for me): I'll have to maintain another server, although point 4 makes it kinda futile to use a separate server

The more I think about it, the more I think I shouldn't bother creating this.

[joker_josue]

Ninja's don't need sleep, I do

They really don't sleep! I had already noticed that...

A question for the OP:
@Vod What was the practical use you had in mind for this type of solution?

[PowerGlove]

The more I think about it, the more I think I shouldn't bother creating this.

Just a heads-up in case you do decide to spend time on this: I'm probably going to tackle adding native PM-exporting support to SMF sometime this year. I haven't got the details fully worked-out yet, but it's likely I'll add support for multiple file-formats (mbox included) and probably look into some kind of authenticity-proving mechanism, too.

[LoyceV]

I'm probably going to tackle adding native PM-exporting support to SMF sometime this year. ~ and probably look into some kind of authenticity-proving mechanism, too.

If it includes proving something, that would be very, very cool! It could be as simple as a checksum somewhere on the forum, like bitcointalk.org/pm.php?action=10924810. All it needs is to show a checksum. But that won't work once the PM is deleted.

[Vod]

@Vod What was the practical use you had in mind for this type of solution?

I mentioned in my last post that I know there is a lot of valuable data people want to keep in those messages.  In case something happens to those messages, it would be nice to have a backup.

You can prove authenticity of any PM, even after deleting it from your inbox, to anyone who trusts me (and my bot). You'll have a file, I'll have a checksum. The checksum doesn't even need to say who it's from, that will be in the file.

You could go a step further:
Allow a person to then edit the PM/file, redacting what they do not want public.  They could use a strikeout or some other code.  Then they repeat the process with you with a different option, or forward the PM to your bot, and you reparse.   You remove all the strikethrough, and compare the checksum.  If it's valid, you can re-save the file without the strikethrough and post the new checksum.

You could make a commercial enterprise out of this.  Part 1 module for bitcointalk and Part 2 core redaction code.  It can even be secured further by using a cloud service so everything is alway encrypted even to you and your bot - no trust necessary.  

[LoyceV]

Allow a person to then edit the PM/file, redacting what they do not want public.  They could use a strikeout or some other code.  Then they repeat the process with you with a different option, or forward the PM to your bot, and you reparse.   You remove all the strikethrough, and compare the checksum.  If it's valid, you can re-save the file without the strikethrough and post the new checksum.

That could work, but it makes it easy to take things out of context. See what's left if you remove the strike through text here.

You could make a commercial enterprise out of this.  Part 1 module for bitcointalk and Part 2 core redaction code.  It can even be secured further by using a cloud service so everything is alway encrypted even to you and your bot - no trust necessary.  

I wouldn't know how to create a bot that handled text without being able to read the text.

[Vod]

Allow a person to then edit the PM/file, redacting what they do not want public.  They could use a strikeout or some other code.  Then they repeat the process with you with a different option, or forward the PM to your bot, and you reparse.   You remove all the strikethrough, and compare the checksum.  If it's valid, you can re-save the file without the strikethrough and post the new checksum.

That could work, but it makes it easy to take things out of context. See what's left if you remove the strike through text here.

Redacted documents are used in court regularly without such fear.  Your example could come out as:
********** work, but ************************************************** strike ******** here.
not
work, but strike here.

[dkbit98]

Didn't hard test it, so idk... Worked for my inbox/outbox.

Well done!
That was ninja speed fast.
I can only imagine what you can make when you complete hard test after good night sleep. 

Just a heads-up in case you do decide to spend time on this: I'm probably going to tackle adding native PM-exporting support to SMF sometime this year. I haven't got the details fully worked-out yet, but it's likely I'll add support for multiple file-formats (mbox included) and probably look into some kind of authenticity-proving mechanism, too.

You probably won't believe this, but after noticing TryNinja posting his quick solution exporting PM's, I had a though that he should join forces with you aka PowerGlove, and that would be a true bitcointalk Power couple  
Best (public) devs this forum had ever since I joined the forum.

[PowerGlove]

It could be as simple as a checksum somewhere on the forum, like bitcointalk.org/pm.php?action=10924810. All it needs is to show a checksum. But that won't work once the PM is deleted.

Yup, something like that occurred to me, too. But, instead of calculating/retrieving a cryptographic hash based on some PM-identifier, it would instead confirm that a given hash is good (that is, that a PM with that hash did exist in the database at some point). There would basically only need to be one new database table that gets a hash appended to it every time someone sends a PM (and that table wouldn't be affected by people purging/deleting their PMs). There's some tricky stuff to consider about how exactly the hash should be calculated (some normalization stuff, and packing the subject and the UIDs of the sender and recipient(s) into the text to be hashed, etc.) but I don't see any real blockers (I mean, don't take that to the bank: I only really thought about this for ~2 hours something like 3 months ago, so when I actually sit down to code it I may bump into things that I either haven't considered yet or that I've since forgotten about).

Handling redaction (that is, confirming the authenticity of arbitrary fragments of PMs) messes up the simplicity of the above approach, and makes things pretty gnarly. I can think of a handful of ways to make it work, but one non-technical concern I have is people building cases against each other based on "verified" but out-of-context and/or intentionally-misleading PM fragments. On that basis, I'm tempted to avoid the redaction quagmire altogether and just stick to a simple all-or-nothing approach to proving PM authenticity.

I've had more recent ideas like adding a new BBCode tag (named vquotepm, or something) that would render in a different style/color compared to regular quote tags and that would be used to quote "verified" PMs (verified in the sense that the server confirmed the quote's correspondence-to-source before embedding a digital signature to that effect).

(And finally, my most recent thought is to not do any of the above, and just fold PM-authentication into PM-exporting: that is, when I get around to tackling that, I'll look into ways to allow the download of some kind of server-signed authenticity-proving data, too.)