64k Character Limit

[DYING_S0UL]

I'll try to keep it as short as possible, and lock the thread as soon as I get a proper answer.

1. Before making a very long post, do you guys ever calculate whether you are about to hit the 65535 bytes / 64k character limit for post?
If yes, then how? With what tool? Some of you guys have very long posts, so I assumed you guys knew.

2. Is BBcodes itself (url, img, table etc) are also part of this calculations? Or only the things we see after making the post is consider to be part of that 64k char limit?

3. My language 'Bangla" takes more space than "English", so I'll usually be hitting the limit way sooner than others.
And I was wondering if anyone could calculate and give me the exact character limit for my specific language? Before it gets broken at somewhere at the end or into two parts It has something to do with UTF-16 or UTF-8.

Assuming if BBcodes are not part of this big calculation.

[khatarnak]

AFAIK it is 64k bytes, not characters, so yeah, Bangla will hit the limit sooner than English.
There is not a fixed character limit for Bangla since the limit is in bytes. The UTF-8 byte count of the actual text is what matters.

I guess these resources can not help you entirely but partially.

• [GUIDE] Bitcointalk forum etiquette
• SMF User Help: Posting (see SMF Bulletin Board Code)
• [LEARN] BBCode Lessons & Tutorials [+tutorial videos!]
• Instructions on how to use BBCode

It has a few useful links about posting and BBCode.

[ryzaadit]

1. Before making a very long post, do you guys ever calculate whether you are about to hit the 65535 bytes / 64k character limit for post?
If yes, then how? With what tool? Some of you guys have very long posts, so I assumed you guys knew.

Either with Extension or Copy your post to word counter. With Google Extension: Word Counter, you can just CTRL + A = SELECT All > Right Click > Choose Count Word & Character.

2. Is BBcodes itself (url, img, table etc) are also part of this calculations? Or only the things we see after making the post is consider to be part of that 64k char limit?

Yes

[Rhow]

2. Is BBcodes itself (url, img, table etc) are also part of this calculations?

Yes.....

AFAIK it's 64,000 characters. I tested it recently and it hasn't changed.

One more question. On this 64k characters, are there counting tags char. also? like [color=black ... /color]

(Again) AFAIK yes, each and every character you use in the post's / topic's textbox is counted towards the 64K limit.

Or only the things we see after making the post is consider to be part of that 64k char limit?

Since theymos said it's byte-based, I think the character-based calculation may not be reliable.

Posts over 65535 bytes get cut off at 65535 bytes. If that makes the BBcode invalid in certain ways, then you get the INVALID BBCODE error.

Also, note that a byte is different from a character. Because SMF stores "special" characters in a particularly space-inefficient way, each Russian character uses about 6 bytes, and each line break also uses 6 bytes.

[PX-Z]

I'll try to keep it as short as possible, and lock the thread as soon as I get a proper answer.

1. Before making a very long post, do you guys ever calculate whether you are about to hit the 65535 bytes / 64k character limit for post?
If yes, then how? With what tool? Some of you guys have very long posts, so I assumed you guys knew.

Haven't experience yet, so i cant tell, loyce can answer your question since he made longs posts with tables from his archives, see quoted post below.

2. Is BBcodes itself (url, img, table etc) are also part of this calculations? Or only the things we see after making the post is consider to be part of that 64k char limit?

Yes, bbcodes, images, etc. everything that you input on the textarea field since that character or saved on db probably on text data type which only accepts roughly 64kb

The limit per post is about 64 kB, although I've hit lower limits when entering lots of BBCode (I think it was a table).

3. My language 'Bangla" takes more space than "English", so I'll usually be hitting the limit way sooner than others.
And I was wondering if anyone could calculate and give me the exact character limit for my specific language? Before it gets broken at somewhere at the end or into two parts It has something to do with UTF-16 or UTF-8.

Assuming if BBcodes are not part of this big calculation.

I wonder character counter browser extensions can calculate the characters on bangla language too.

[DYING_S0UL]

I wonder character counter browser extensions can calculate the characters on bangla language too.

I was wondering the same thing whether this word counter tool can calculate languages other than English.

So I had a translation post that I needed to divide into two parts before posting. And I just tested it with this tool and it says 1,829 words 24,594 characters. It's nowhere near the 64k char limit.

Note that, that post had a lot of mixed characters (English + Bangla + lots of BBcodes)

Yes, bbcodes, images, etc. everything that you input on the textarea field since that character or saved on db probably on text data type which only accepts roughly 64kb

Now how do I calculate whether it crossed 64kb again?

Ok, then i saved everything in a .txt file (UTF-8), and the file size was 35.1 KB (36,023 bytes)
If I save it under UTF-16 LE, it takes 49.6 KB (50,850 bytes)

2. Is BBcodes itself (url, img, table etc) are also part of this calculations? Or only the things we see after making the post is consider to be part of that 64k char limit?

Yes

Thank you. One confusion is now clear.


To all,
I think it makes sense now. I'm almost close to that 64kb size with UTF-16 LE, and I remember since my post got divided, I intentionally did not used all of the allowed characters. Instead I tried to split the post evenly, and made the second post. Both were perfect and wasn't broken.

[hd49728]

I'll try to keep it as short as possible, and lock the thread as soon as I get a proper answer.

1. Before making a very long post, do you guys ever calculate whether you are about to hit the 65535 bytes / 64k character limit for post?

No, I don't calculate that and don't care to check that.

If you are fearful that it can lead to an error, you can compose your post with a document tool, save it as a draft, before copying, pasting and posting it in the forum. If the error appears, you can return to your draft, split it into two parts and try to post again.

What Happen with my post?
There are any words limitations rather than English language ?

Yes there's a limit/post as per theymos its 65535 bytes (65,535 ASCII/English characters).  But when it comes to other languages like Arabic/Urdu it needs Unicode to save and size of it depends on the encoding for example:

UTF-16 - Each Character needs two-bytes
UTF-32 - Each Character needs four-bytes

Edit: I believe it's UTF-32 which means 4-bytes/character, I tested it in a PM (assuming PM/Posts have same character limit) and I'm able to send a big message with mix Urdu/English characters and it automatically got trimmed at 21,799.

You can use this online counter, choose the language too.
https://www.charactercountonline.com/

[DYING_S0UL]

If you are fearful that it can lead to an error, you can compose your post with a document tool, save it as a draft, before copying, pasting and posting it in the forum. If the error appears, you can return to your draft, split it into two parts and try to post again.

Not always though! Only when I'm making a very longggggg translation post

Yes there's a limit/post as per theymos its 65535 bytes (65,535 ASCII/English characters).  But when it comes to other languages like Arabic/Urdu it needs Unicode to save and size of it depends on the encoding for example:

UTF-16 - Each Character needs two-bytes
UTF-32 - Each Character needs four-bytes

Edit: I believe it's UTF-32 which means 4-bytes/character, I tested it in a PM (assuming PM/Posts have same character limit) and I'm able to send a big message with mix Urdu/English characters and it automatically got trimmed at 21,799.

You can use this online counter, choose the language too.
https://www.charactercountonline.com/

According to Google, "Bangla" uses Unicode as its character standard and UTF-8. I guess it's pointless now, I'll just have to keep guessing, and believe in my gut, and pray that I don't hit the limits.
Cause my post contains both English Character (the BBcodes), and my local language. So the whole calculation thing becomes messy.
Also, I'll hit the post size limit 64kb, long before I hit the textArea 64k character limit, as "Bangla" language takes 3 bytes per character, 3x than English

[PX-Z]

Now how do I calculate whether it crossed 64kb again?

Ok, then i saved everything in a .txt file (UTF-8), and the file size was 35.1 KB (36,023 bytes)
If I save it under UTF-16 LE, it takes 49.6 KB (50,850 bytes)

Take note that saving it to a .txt file first, it will add some 0.xx size, just like if you create a blank .txt file.

You can use this tool, https://charactercounter.com/byte-counter
It shows character count and the total bytes either in utf-8 or 16 and 32

[DYING_S0UL]

Now how do I calculate whether it crossed 64kb again?

Ok, then i saved everything in a .txt file (UTF-8), and the file size was 35.1 KB (36,023 bytes)
If I save it under UTF-16 LE, it takes 49.6 KB (50,850 bytes)

Take note that saving it to a .txt file first, it will add some 0.xx size, just like if you create a blank .txt file.

You can use this tool, https://charactercounter.com/byte-counter
It shows character count and the total bytes either in utf-8 or 16 and 32

With UTF-8



With UTF-16



With UTF-32



Tested against a translation post of mine and this is what I got. Anyway, thank you for the link, it works, and the numbers are correct it seems, as every character are being calculated separately (English, Bangla, special char I meant).

Lastly I am locking this thread. I don't think I need to keep it open anymore, as all of my questions were answered. Thank you guys for all the helps.  

[theymos]

Checking it with standard UTF encodings won't work. Here is how this post is stored internally:
https://bitcointalk.org/misc/encoding_67038390.txt

It's on this format that the 64kiB limit is applied. There are a bunch of quirks in how this format works, but most importantly:
 - Characters not in ISO-8859-1 are converted to numeric HTML entities
 - Newlines become <br />
 - & becomes &amp;, " becomes &quot;, ' becomes &#039;, < becomes &lt;, and > becomes &gt;
 - If there's more than one space in a row, all but the first are converted to &nbsp;