Data-stream encryption

[Anti-Cen]

I rolled my own encryption regardless of all the advice not too (they would say that !) because I work on windows and will not trust any
of the Microsoft's black-box encryption tools and passing keys to Microsoft for AES encryption is a big red flag for me.

What i need is fast encryption for a Tor style network that will work with data streams for sending large volumes of data
but the trouble is the data might be send/received in random sizes and i want to relay the data as soon as it comes in

incoming upstream node [------300k-----] [-----200k-----][-----67k-----]
receiving local node        [------200---][----467k--------------------------]

So i can swap all "A's" to become "Z's" and "Z''s" becomes "A's" based on a secret key used to mix the data up
but this encryption would be easy to crack and because of the speed I don't want to keep inserting points into the
byte array being encrypted.

Now i could added extra shift into byte codes based on looking for the letters 'HTTP' but the trouble is that the
data could arrive in a blocks starting with 'TP' or ending in 'HT'  so it would be a bit hit and miss and I don't even know
the type of data that will be sent because the nodes act as SOCK4/5 and HTTP Proxy servers which you guys might like
for running nodes on Bitcoin.

So far my code is sweet but generating the CypherCode is slow so it's cached and is returned from MakeCypherCode as a 512 byte array
and the speed is second to none.

Code:

public static byte[] DecryptFast(byte[] Text, string Key)
       {
           byte[] CypherCode = MakeCypherCodes(Key);
           for (uint f = 0; f <= Text.Length - 1; f++)
           {
               Text[f] = CypherCode[Text[f]+256];
           }
           return Text;
       }

       public static byte[] EncryptFast(byte[] Text, string Key)
       {
           byte[] CypherCode = MakeCypherCodes(Key);
           for (uint f = 0; f <= Text.Length - 1; f++)
           {
               Text[f] = CypherCode[Text[f]];
           }
           return Text;
       }

HTTPS traffic is already semi safe on this network because the CONNECT request that includes the domain name is encrypted before being sent but
the certificate response could still be sniffed but after that I plan on just sending the data "as is" since it's already encrypted

Too be honest I don't think there is a real good solution to this problem but i thought I would try anyway.
 

[1715230148]

1715230148

[1715230148]

1715230148

[1715230148]

1715230148

[cellard]

If you use Windows and don't trust AES what about Veracrypt? you can use cascaded encryption settings that not include AES, you got Whirpol, Serpent, Twofish... anyway, why even use Windows? it's reasonable to assume that you are compromised by default using Windows to be honest.

dm-crypt and LUKS as Linux options but if you still need Windows then Veracrypt has insant on-the-fly decryption but im not sure how I would even go around adding that into a onion style network. I was about to make a thread about how attackers will try to incriminate people running nodes by claiming the blockchain has illegal data on it such as links to drug and CP sites and other crap which would mean anyone hosting a full node would by default be a criminal... which is why I wanted a forum dedicated to encryption only, I think it will be a must in the future for people running nodes specially, just be sure to do it right, I bricked a couple of hard drives testing things out, my password doesn't work, luckily I don't have anything of value in there.

Also please check this thread.

[Anti-Cen]

Also please check this thread.

Top notch man and i will check "Veracrypt" out if I can get the source code and not just a Dll plus it's not bloated and
runs fast then I might use it.

The trouble is you can find lots this type of code but it's all Linux based and I manged to rip out Bitcoins encryption that
I trust for key exchange but it's much too slow for bulk data and "on-the-fly" encryption.

I was about to make a thread about how attackers will try to incriminate people running nodes by claiming the blockchain has illegal data on it

Building a network and trying to address all the problems created by criminals is a problem and on one hand we have microsoft scum
and it's new user agreement.

"The lengthy Services Agreement warns against sharing inappropriate content, including 'nudity, bestiality, pornography, offensive language, graphic violence, or criminal activity.'

acting like big brother and on the other hand we have homosexual sex perverts using the network so when it comes to privacy it's becomes
all or nothing and I fear the fascist state doing more damage than the perverts myself.

Tor with just a few hundred exit nodes and slow speeds has not really advanced much in ten years and all the exit node address are known
plus onion sites only work if the server stays connected and on-line so i am trying to to develop something similar to smart contracts but instead of
just hosting bean counters the nodes host onion type sites instead

Also please check this thread.

Been cutting too much code and putting it in the bin so i missed the post but will pop over now
 

[Colorblind]

Kudos for being extra paranoid.

On the side note why not use some known open-source encryption algo?

[AdSkull89]

Kudos for being extra paranoid.

On the side note why not use some known open-source encryption algo?

Yeah something like triple DES should suit the needs of most paranoid person on earth https://en.wikipedia.org/wiki/Triple_DES

it also free, opensource and easy to implement.

[Anti-Cen]

Kudos for being extra paranoid.

On the side note why not use some known open-source encryption algo?

This is not so easy as you think on windows and I want the code in C# and not as a Dll since I want my program
to work as a single stand alone .exe

I am not after bloatware where you need 5,000 lines of code, it must be fast and work on the fly so it can be
used with a socket stream so that movies and big files can be streamed across an unreliable network.

Just buffering up the data before sending it back in a fixed sized blocks on the server nodes is not really an option
and even then the client might not receive the block on the network as a full block so this is why I am after on-the-fly
encryption.

[Anti-Cen]

Yeah something like triple DES should suit the needs of most paranoid person on earth https://en.wikipedia.org/wiki/Triple_DES

it says

I.e., decrypt with K3, encrypt with K2, then decrypt with K1.
Each triple encryption encrypts one block of 64 bits of data.

performance wise it's going to be too slow compared with

public static byte[] EncryptFast(byte[] Text, string Key)
       {
           byte[] CypherCode = MakeCypherCodes(Key);
           for (uint f = 0; f <= Text.Length - 1; f++)
           {
               Text[f] = CypherCode[Text[f]];
           }
           return Text;
       }

I could do something using BigInts with a bit of maths on 64 bits of data but it needs to stream
and the CPU would be running at a high rate and inserting markers into a byte array means copying the
array to make the space needed so again that's not an option.

Will keep banging away and let you know if I find a solution 

[bob123]

Instead of creating a (very) simple encryption by yourself, which is not just unsafe but also easy to 'crack', you might consider using an known (and trusted) algorithm.
Encryption through substitution and a few linear(!) functions can always be reversed without to much effort.

If you are not trusting windows, why not simply switch to linux? You don't even have to switch completely. You can just use a dual boot system.

Even if you want to stay on windows, it would be way easier and safer to just use an open source implementation of a well tested encryption algorithm.
You can even implement an encryption yourself. But you should really use an known (and tested) encryption algorithm.

This eliminates the risk of any closed source software / malicious implementations without exposing yourself to the risk of an easily reverted encryption.

[Anti-Cen]

Instead of creating a (very) simple encryption by yourself, which is not just unsafe but also easy to 'crack'.
Encryption through substitution and a few linear(!) functions can always be reversed without to much effort.

Encryption is no use if it's too slow and in my case cannot be streamed and unfortunately for the PTB
I tend to trust my own judgment.

If you are not trusting windows, why not simply switch to linux? You don't even have to switch completely. You can just use a dual boot system.

Yes Linux is better but I am too much of an expert with windows and the bloated .NET framework to just jump ship, too old too.

This eliminates the risk of any closed source software / malicious implementations without exposing yourself to the risk of an easily reverted encryption.

My code would be open source and must stand up to peer review and I am sure no one would say a world if i just pasted the AES keys over to
microsoft to process the encryption because that's just whats happen already with windows based wallet on Bitcoin so please don't knock me for
at least trying something new.

here is what I am trying to deal with

Browser --------Entry-Node------Relay-Node--------Exit-Node-------Web-site

The trouble is some web-sites use chucked data and pages are not simply send as <HTML>........</HTML>
but are sent as 'Chunks' for the browser to process and waiting for the exit-node to collect and then encrypt the
whole page is not an option and you have to bear in mind that the entry node might no receive the data as one
big long packet.

Throw in that nodes are stateless apart from knowing the key and you start to see what I am up against and i just tried
putting in a few flag switches on the encryption but that didn't work because packets might be sent out as
[------100k-----------][------20k-----][--------50k--------]
but could be received as
[--1k---][---2k---][----something k-----------------------]

lucky I already have good encryption working on the HTTP request headers used to set up the circuit
but i don't think much more can be done on the payload of the page without breaking some pages so it's
a toss up between good encryption and broken pages as I see it.

1GB encryption/decrytion in under ten seconds is not something that I can just pick up off the shelf or I would believe me.

[bob123]

I tend to trust my own judgment.

A lot of people do that. And the majority get burned.
Probably the best example: Brainwallets.

Yes Linux is better but I am too much of an expert with windows

 

This eliminates the risk of any closed source software / malicious implementations without exposing yourself to the risk of an easily reverted encryption.

My code would be open source and must stand up to peer review

I guess you didn't understand my statement.
The point is that it is senseless to create an 'encryption' which can easily be reverted.

and I am sure no one would say a world if i just pasted the AES keys over to
microsoft to process the encryption because that's just whats happen already with windows based wallet on Bitcoin

What are you talking about? 

so please don't knock me for at least trying something new.

I did not intend to 'knock' you.
I just tried to show you why it is a bad idea to 'create' an own 'encryption' algorithm.

You seem to lack the basic knowledge about cryptography.
Substitution is an easily breakable approach (i.e. known-plaintext attack / frequency analysis).
Even in combination with linear functions this can not be considered safe. In no way.
Your 'encryption' will be attackable without a big effort.

[Anti-Cen]

A lot of people do that. And the majority get burned.
Probably the best example: Brainwallets.

I am not anyone and ho fucking dear, I have seen that name which i will come too.

I guess you didn't understand my statement.
The point is that it is senseless to create an 'encryption' which can easily be reverted.

I don't think I have many other options, encryption is ten a penny but unfortunately not just anyone will do.

What are you talking about? 

I pulled the bitcoin Secp256k1 from a windows library and I use that for key exchange because it is second to none
but after that the code in Bitcoin uses AES for the signature from what I had seen. Here is the code that's in the project I pinched the Secp256k1 from.

Code:

private byte[] DecryptData(byte[] Key, byte[] IV, byte[] cipher)
        {//RijndaelManaged AES decryption
            aesEncryption.IV = IV;
            aesEncryption.Key = Key;
            ICryptoTransform decryptor = aesEncryption.CreateDecryptor();
            byte[] decryptedData = decryptor.TransformFinalBlock(cipher, 0, cipher.Length);
            return decryptedData;
        } 

I could dig deeper into this but here is another bit of code from the same project !

            ECEncryption encryption = new ECEncryption();
            byte[] encryptedP = encryption.Encrypt(Settings.PrivateKey, publicKey, message);
            byte[] decryptedP = encryption.Decrypt(Settings.PublicKeyPoint , encryptedP);
            string decryptedMessageP = Encoding.UTF8.GetString(decryptedP);

            byte[] encrypted = encryption.Encrypt(Settings.PublicKeyPoint, message);
            byte[] decrypted = encryption.Decrypt(Settings.PrivateKey , encrypted);
            string decryptedMessage = Encoding.UTF8.GetString(decrypted);
            MessageSignerVerifier messageSigner = new MessageSignerVerifier();
            SignedMessage signedMessage = messageSigner.Sign(privateKeyBig, "Test Message to sign, you can verify this on http://brainwallet.org/#verify");
            bool verified = messageSigner.Verify(signedMessage);

The header for encryption reads

public class ECEncryption
    {
        private readonly ECDsaSigner signer = new ECDsaSigner();
        private ECElGamal ecElGamal = new ECElGamal();
        private RijndaelManaged aesEncryption = new RijndaelManaged();
        private RNGCryptoServiceProvider rngCsp = new RNGCryptoServiceProvider();.............

This is why I stated that Bitcon uses AES after key exchange but now I have to question this myself
but this project I pulled this code from is quite a common one so I am not sure what to make of this.

[bob123]

I pulled the bitcoin Secp256k1 from a windows library and I use that for key exchange because it is second to none
but after that the code in Bitcoin uses AES for the signature from what I had seen. Here is the code that's in the project I pinched the Secp256k1 from.

Code:

private byte[] DecryptData(byte[] Key, byte[] IV, byte[] cipher)
        {//RijndaelManaged AES decryption
            aesEncryption.IV = IV;
            aesEncryption.Key = Key;
            ICryptoTransform decryptor = aesEncryption.CreateDecryptor();
            byte[] decryptedData = decryptor.TransformFinalBlock(cipher, 0, cipher.Length);
            return decryptedData;
        } 

Bitcoin does not use AES for signature. AES is used for symmetric encryption.
Bitcoins signature algorithm is the ECDSA (DSA on elliptic curve). And Secp256k1 refers to the curve which is used.

This is why I stated that Bitcon uses AES after key exchange but now I have to question this myself
but this project I pulled this code from is quite a common one so I am not sure what to make of this.

Besides AES, bitcoin also doesn't have any key exchange.
An key exchange is required when you have to actually exchange a symmetric encryption/decryption key.
Asymmetric encryption made that obsolete with the drawback of being significantly slower.
But the bitcoin protocol does not include any (encrypted) information exchange via the network.

[2112]

1GB encryption/decrytion in under ten seconds is not something that I can just pick up off the shelf or I would believe me.

You are neglecting a whole branch of cryptography: https://en.wikipedia.org/wiki/Stream_cipher .

The shelf is reasonably well stocked, recent overview: https://en.wikipedia.org/wiki/eSTREAM .

[Anti-Cen]

Bitcoin does not use AES for signature. AES is used for symmetric encryption.
Bitcoins signature algorithm is the ECDSA (DSA on elliptic curve). And Secp256k1 refers to the curve which is used.

Yes i am starting (I think) to get the picture and the code I got this impression from was from here
https://github.com/TangibleCryptography/Secp256k1

Seems to tick all the boxes if you care to look

See second post here https://bitcointalk.org/index.php?topic=2826752.msg%msg_id%

I think this code was hanging around in the forum from about 2014
https://bitcointalk.org/index.php?topic=574882.0;prev_next=prev

The ECPOINT in his code uses key exchange from Secp256k1 for AES key/iv as you can see so
I am not sure whats going on here so maybe you can cast some light on this and excuse me for
saying that Bitcoin used AES

I started to suspect something might be wrong (late in the day) when I noticed the name "Brainwallets"
and I would give you a merit for correcting me but I don't seem to get many to give away around here so
it will just have to be a kiss 

[andrew1carlssin]

1GB encryption/decrytion in under ten seconds is not something that I can just pick up off the shelf or I would believe me.

You are neglecting a whole branch of cryptography: https://en.wikipedia.org/wiki/Stream_cipher .

The shelf is reasonably well stocked, recent overview: https://en.wikipedia.org/wiki/eSTREAM .

Indeed ... I am not in the academic field .. but yes sounds that the branch is huge ...

Lecture 3: Stream Ciphers, Random Numbers and the One Time Pad by Christof Paar
https://youtu.be/AELVJL0axRs

ps-> Christof Paar, textbook is awesome ... worth a try ..

[Anti-Cen]

You are neglecting a whole branch of cryptography: https://en.wikipedia.org/wiki/Stream_cipher .

The shelf is reasonably well stocked, recent overview: https://en.wikipedia.org/wiki/eSTREAM .

Thanks for the link and you gave me the right answer in the end after looking around and finding very little code that would run in windows let alone
in C# that would stream or was open source and would give me any performance let alone not being bloatware that falls back on BouncyCastle or being open source.

you see I can encrypt something in Dot-spyware-net in about five lines of code but i don't trust them so what i think I might do is to semi encrypt
the stream, chuck it to Microsoft's black box code after reversing the key or something and then reverse the process during decryption.

I need to do some banchmark testing but why re-invent the wheel when only a spoke is broken and I might even throw in some compression
whilst I am about it.

what does concern me is the open source Secp256k1 code that I picked up and was planing on using for key exchange because it's starting
to look tainted now and I do confess that I don't really understand the maths behind points on a wave and this bit of code is starting to look
suspect to me.

   public static class Secp256k1
    {
        public static readonly BigInteger P = "FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F".HexToBigInteger();
        public static readonly ECPoint G = ECPoint.DecodePoint("0479BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798483ADA7726A3C 4655DA4FBFC0E1108A8FD17B448A68554199C47D08FFB10D4B8".HexToBytes());
        public static readonly BigInteger N = "FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141".HexToBigInteger();
    }

Does anyone recognize these hard coded strings ?

[andrew1carlssin]

Sorry my layman question ... but it came to my mind ...

stream ciphers encrypts bit by bit ... how does it relates to CPU clock ?

[bob123]

The ECPOINT in his code uses key exchange from Secp256k1 for AES key/iv as you can see so
I am not sure whats going on here so maybe you can cast some light on this

I am not sure what code you are looking at.
The ECPoint you have linked (https://github.com/TangibleCryptography/Secp256k1/blob/ee6ddaa59d3c61295bb24bcc903bb16fb528b933/Secp256k1.Core/ECPoint.cs) consists of this:

Code:

public ECPoint(BigInteger x, BigInteger y, bool isInfinity)
        {
            _x = x;
            _y = y;
            _isInfinity = isInfinity;
        }

Feel free to directly link to the AES function.
But as i have already mentioned, AES is used for symmetric encryption.
The bitcoin protocol doesn't require anything to be encrypted.

While signing message/transaction is theoretically like encrypting (with private- and public key swapped), there is no AES (or anything comparable) used.

You also seem to be a little bit confusing with your term of 'key exchange'.

I'd suggest you read this: https://en.wikipedia.org/wiki/Key_exchange


Regarding the IV of (any) AES encryption:
The IV can be whatever you want. So it seems that someone just decided to use parameters from the secp256k1 as IV.

Read here for more information: https://en.wikipedia.org/wiki/Initialization_vector

    public static class Secp256k1
    {
        public static readonly BigInteger P = "FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F".HexToBigInteger();
        public static readonly ECPoint G = ECPoint.DecodePoint("0479BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798483ADA7726A3C 4655DA4FBFC0E1108A8FD17B448A68554199C47D08FFB10D4B8".HexToBytes());
        public static readonly BigInteger N = "FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141".HexToBigInteger();
    }

Does anyone recognize these hard coded strings ?

What do you mean by 'recognize' ?
You should read my answer to your previous post more carefully:

Bitcoins signature algorithm is the ECDSA (DSA on elliptic curve). And Secp256k1 refers to the curve which is used.

Those are the values for the secp256k1 curve.

Look here: https://en.bitcoin.it/wiki/Secp256k1

[Anti-Cen]

Feel free to directly link to the AES function.

https://github.com/TangibleCryptography/Secp256k1/blob/master/Secp256k1.Core/ECEncryption.cs

The ECPoint gets encoded with with a AES key and IV

Code:

 public byte[] Encrypt(ECPoint publicKey, byte[] data)
        {
            byte[] key;
            var tag = ecElGamal.GenerateKey(publicKey, out key);
            var tagBytes = tag.EncodePoint(false);

            byte[] iv = new byte[16];
            rngCsp.GetBytes(iv);

            aesEncryption.IV = iv;

            aesEncryption.Key = key;

            ICryptoTransform crypto = aesEncryption.CreateEncryptor();

Later these are used for the signature signing

Those are the values for the secp256k1 curve.

So the hard coded value seem to be correct and I think I had looked in to this before so I just took it that the rest of the
code would work with bit-coin since it contained the Base58 and Hash160 code but somewhere along the way it seems
like it come off the rails

[Anti-Cen]

stream ciphers encrypts bit by bit ... how does it relates to CPU clock ?

That sounds like how many machine code instruction cycles does it take to encode each bit which is
related to the speed of the CPU so you want a low number of cycles and a high clock rate to get any speed

[Anti-Cen]

in my quest to find what i was looking for I took the code here, nice sweet and simple it is and it used maths.mod to encrypt data
and I got it up and running in seconds ready for a bit of benchmark testing on 1GB of data (1mb blocks).

https://www.codeproject.com/Articles/17703/Stream-Based-Encryption-for-NET

I stopped the program after about sixty seconds and it had only managed to encrypt/decrypt
500mb of data so i think you can have good encryption or fast performance but you cannot have both.

BigINT are useful for encryption but again I just cannot get the performance I want using this method.

My current plan is to semi encrypt the data, tweak the key and pass it over to Microsofts black box
encryption but I won't be using AES encryption because the .NET version leaks memory which you only get
to see if you call it half a million times or so which might sound a lot but a proxy server can reach these
type of number in little over an hour.

 

[Ix]

To send encrypted data over a vulnerable protocol (like the internet) you need:

1) key exchange (diffie-hellman usually)
2) a cipher (like salsa20 + initialization vector per chunk)
3) data integrity checks/authentication (hmac / poly1305)

You generally need to use TCP and you need a small maximum message size (chunk) - around 8kb or so typically - otherwise you have to resend huge chunks in case of corruption. You can use UDP which it sounds like you want, but then you need to reinvent a lot of TCP on top of UDP.
You can't use some hardcoded key because then anyone can look at your software and decode all communications. You need a different IV per message or observers can easily determine your key. You need authentication of the data or you will not know if the peer sent bad info or if it was accidentally/maliciously changed in transit.

The NaCl and LibSodium libs implement most of this for you in "secretbox". It's also available in the Go "x/crypto" library. There might be a C# implementation, I don't know.

[cellard]

satoshi was a Windows developer, the early Bitcoin releases were only released in Windows... so don't underestimate people that are good coders in Windows, as long as the project is open source and other developers can keep developing in Linux later on as well then it should be good. I recommend that people learn programing in Linux for projects of such nature tho.

Isn't AES used for the wallet.dat encryption? I would like more options for the encryption of the wallet.dat file, give us the ability to select cascaded encryption Veracrypt style and the option to mask the funds too so you need to enter the password to see the funds.. anyway this is off topic I will create a thread later.

[Anti-Cen]

I don't think people are quite picking up the point here or maybe I didn't put it across correctly but we are not talking about
encrypting small amounts of data like a 2k whats-app message but something more like 100k needed for the HTML on a web-page
and then six javascript files within the page followed up by say ten 2mb .PNG/Gif images files and a CSS file

Performance is critical here because if these requests are not serviced within 2-5 seconds then the browser will try sending the request again
using multi threaded requests so both the client and server nodes need to also be multithreaded also and the exit nodes need to start pumping the data
back across the network even before they have the whole reply from the web-site.

Things are made worse because often data is chunked which more or less means streamed and you have relay nodes possibly in the middle
plus having to service 100 requests all at the same time so buffering up the data to be encrypted in memory and keeping much state information
simply is not an option.

Data can be sent over TCP/IP as a 30K message but can be received as three 10k sections with a second delay between them all
but you still have to start feeding the web-browser as soon as you receive the first 10k or else it might timeout so I have no option
but to use on-the-fly stream encryption and it must perform like Greece lightning.

What i do know is that Microsoft AES encryption works nearly as fast as I can iterate over a byte array
using un-safe code from C# with C++ style pointers so I don't know if specialist hardware is being used to
get the performance for AES encryption but that's the type of speed that I need and using Maths.Mod or BigINT's just won't
give it to me

Microsoft's AES encryption memory leaks and I am not trusting back-door bill gates with any keys or data given that
I don't know whats being copied and uploaded back to the NSA/CIA servers all night long.

To send encrypted data over a vulnerable protocol (like the internet) you need:

1) key exchange (diffie-hellman usually)
2) a cipher (like salsa20 + initialization vector per chunk)
3) data integrity checks/authentication (hmac / poly1305)

Yes got safe key exchange using Bitcoins Secp256k1 on a control channel and the data-streams have good encryption
on the HTTP Requests send to the server nodes but these requests are only between 400-2000 bytes in size so no trouble to encrypt.

You generally need to use TCP and you need a small maximum message size (chunk) - around 8kb

8K chunks are no use when
sending 2mb images over the internet and often using relays, not going to work that's not.

You can use UDP which it sounds like you want, but then you need to reinvent a lot of TCP on top of UDP.

Google is trying to use UDP instead of TCP/IP for web-pages with the Chrome browser but I am not doing anything apart from
DNS using UDP:53

I keep the control channel and a number (Six just now) of these bi-directional streams open per client so that a node sitting behind a firewall
can still act as a server but in general the number of streams needed to service a web-browser can reach as high as 30 steams at a time

[Ix]

8K chunks are no use when
sending 2mb images over the internet and often using relays, not going to work that's not.

Sure it will, and it will work much better than attempting to encrypt it all in one chunk. The maximum packet size is around ~1500 bytes as it is. If you send a 2MB file and one bit gets flipped somewhere along the way, without chunking the entire thing will have to be resent. There is very little overhead to doing this.

It sounds like you want to do something similar to Tor, maybe you should investigate what it does?

[Anti-Cen]

Sure it will, and it will work much better than attempting to encrypt it all in one chunk. The maximum packet size is around ~1500 bytes as it is. If you send a 2MB file and one bit gets flipped somewhere along the way, without chunking the entire thing will have to be resent. There is very little overhead to doing this.

The lower levels of the ISO reference model deal with bit's being flipped, it happens all the time but all a socket can receive in one chuck is 65535 bytes
so really I am sending images and bigger files just like web-site pump out data and that works.

It sounds like you want to do something similar to Tor, maybe you should investigate what it does?

Yes I know my way around Tor, been an exit node myself before now plus hosted a onion site however it's
slow, has no redundancy to host sites if the server node is down plus a few other little problems like only having a few well known
exit nodes and they are all black-listed plus something I don't want to go into here.

Down the road I will be dealing with Whats-app type messages, file sharing, hidden sites but most of it's based on end to end encryption
using a public key generated for each node without needing any centralization however I don't really think this will be possible if it becomes
popular but at least I admit it up front.

One day people will laugh that they had emails addresses instead of a public key knowing that big brother is reading everything we say.

[Ix]

The lower levels of the ISO reference model deal with bit's being flipped, it happens all the time but all a socket can receive in one chuck is 65535 bytes
so really I am sending images and bigger files just like web-site pump out data and that works.

I'm not sure what you mean by ISO reference model. TCP has a 2 byte checksum which can easily be accidentally fooled. Malicious bridge nodes (in Tor terminology) can easily switch all kinds of bits that TCP cannot detect or they can just modify the checksum to match. That is why you need authentication.

https://en.wikipedia.org/wiki/Maximum_transmission_unit

"For example, a 1500-byte packet, the largest allowed by Ethernet at the network layer (and hence over most of the Internet), ties up a 14.4k modem for about one second. Large packets are also problematic in the presence of communications errors. If no forward error correction is used, corruption of a single bit in a packet requires that the entire packet be retransmitted, which can be very costly."

Now imagine you sent your 2MB image without chunking and you have the same problem as a TCP packet.

[Anti-Cen]

I'm not sure what you mean by ISO reference model.

http://www.itprotoday.com/management-mobility/isoosi-ieee-8022-and-tcpip

TCP has a 2 byte checksum which can easily be accidentally fooled. Malicious bridge nodes (in Tor terminology) can easily switch all kinds of bits that TCP cannot detect or they can just modify the checksum to match.

But 2mb are still sent as 2mb blocks from sockets regardless of the trouble in keeping a thread open to send data in little block or encryption
and the lower levels deal with it most of the time and there is not much i can do about it.

14.4k modem   When i started we only had 1200 baud and being able to send faxes involved an upgrade to the modem and these
things were expensive I can tell you but still it was better than Prestel and it's teletext type graphics.

[Ix]

But 2mb are still sent as 2mb blocks from sockets regardless of the trouble in keeping a thread open to send data in little block or encryption
and the lower levels deal with it most of the time and there is not much i can do about it.

The point is you do have to deal with it with a stream cipher, if you want a protocol that is in any way reliable. TCP checksums are not sufficient, and without chunking and authentication you won't know something is wrong until after the whole file is sent, and then the whole file must be sent again.

[bob123]

The lower levels of the ISO reference model deal with bit's being flipped, it happens all the time but all a socket can receive in one chuck is 65535 bytes
so really I am sending images and bigger files just like web-site pump out data and that works.

But 2mb are still sent as 2mb blocks from sockets regardless of the trouble in keeping a thread open to send data in little block or encryption..

While 65535 bytes is the theoretical maximum for a 'tcp packet', which by the way is stream based (why do you care about packet size when using TCP?),
lower layers are not able to process such big packets without errors.

In the end a single packet sent over a tcp connection is always lower than those 65535 byte.

[Anti-Cen]

why do you care about packet size when using TCP?

I don't really care about the TCP packet size, just the chunk/stream size received for decryption which might not be the same size as what was sent
unless you hang around waiting to see if more data arrives a few seconds later.

This makes encrypting large volumes of data sent over the wire more difficult on top of the performance problem of eating up the CPU  and
frankly I don't have the answer yet.