Bitcoin Forum
November 22, 2026, 12:07:29 PM *
News: Latest Bitcoin Core release: 32.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Improving the word-breaker (SMF patch)  (Read 311 times)
PowerGlove (OP)
Hero Member
*****
hacker
Offline

Activity: 711
Merit: 7437



View Profile
February 17, 2025, 02:54:32 AM
Merited by LoyceV (42), dkbit98 (15), klarki (13), vapourminer (12), ABCbits (9), Pmalek (5), Cyrus (1), yahoo62278 (1), Lafu (1), DdmrDdmr (1), BlackBoss_ (1)
 #1

I ran into a nice suggestion from a long time ago (~12 years):

SMF breaks up long words by inserting a space every 79 characters (it is a space in a <span> with a negative margin). Example: here are 120 'a' characters:

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

When copying/pasting the chars, the space is visible at the 80th position, which is very annoying...

Instead, SMF should insert the standardized <wbr> tag (word break opportunity) already recognized by most browsers. In theory <wbr> is identical to U+200B (ZERO-WIDTH SPACE) but this is false; for example the current Chrome version on Linux (Version 23.0.1271.97) replaces U+200B with '#' when copying & pasting to a non-UTF8 application, whereas <wbr> is nicely invisible...

Maybe newer members haven't encountered this particular issue before, but I know exactly what the above poster is complaining about, and I'm sure that many other people do, too.

Basically, if you (with some exceptions that aren't worth getting into) post an unbroken sequence of 80 or more letters, digits, underscores and/or periods, then it'll automatically get divided into chunks which are each 79 characters in length (or less, in the case of the final chunk).

So, for example, if you were to post the SHA-512 of the text "How would you like to suck my balls, Mr. Garrison?", then instead of it appearing like this:



It would appear like this:



(See that tiny space at around the two-thirds point? Between dbcc and 9e06?)

That "breaker" space will, among other things, cause problems with copy-pasting (because there's now a space in the content that the author didn't intend for there to be), and will also affect double-click selecting, like this:



So, what I've done is add a new kind of "breaker" to SMF (to go along with the three existing variations that are chosen between based on what browser the BBCode parser thinks it's producing markup for). This new breaker avoids the problems described above and is used by default (but, I've left the older breakers accessible by a context variable in case theymos wishes to selectively flip between the new and the old behavior).

Here's the diff:

Code:
--- baseline/Sources/Subs.php	2011-09-17 21:59:55.000000000 +0000
+++ modified/Sources/Subs.php 2025-02-16 23:39:26.000000000 +0000
@@ -1860,24 +1860,27 @@
  if (!empty($modSettings['fixLongWords']) && $modSettings['fixLongWords'] > 5)
  {
  // This is SADLY and INCREDIBLY browser dependent.
  if ($context['browser']['is_gecko'] || $context['browser']['is_konqueror'])
  $breaker = '<span style="margin: 0 -0.5ex 0 0;"> </span>';
  // Opera...
  elseif ($context['browser']['is_opera'])
  $breaker = '<span style="margin: 0 -0.65ex 0 -1px;"> </span>';
  // Internet Explorer...
  else
  $breaker = '<span style="width: 0; margin: 0 -0.6ex 0 -1px;"> </span>';
 
+ if ($context['bbc_use_modern_breaker'] ?? true)
+ $breaker = '<wbr />';
+
  // PCRE will not be happy if we don't give it a short.
  $modSettings['fixLongWords'] = (int) min(65535, $modSettings['fixLongWords']);
 
  // The idea is, find words xx long, and then replace them with xx + space + more.
  if (strlen($data) > $modSettings['fixLongWords'])
  {
  // This is done in a roundabout way because $breaker has "long words" :P.
  $data = strtr($data, array($breaker => '< >', '&nbsp;' => $context['utf8'] ? "\xC2\xA0" : "\xA0"));
  $data = preg_replace(
  '~(?<=[>;:!? ' . $non_breaking_space . '\]()]|^)([\w\.]{' . $modSettings['fixLongWords'] . ',})~e' . ($context['utf8'] ? 'u' : ''),
  "preg_replace('/(.{" . ($modSettings['fixLongWords'] - 1) . '})/' . ($context['utf8'] ? 'u' : '') . "', '\\\$1< >', '\$1')",
  $data);



When doing these sorts of fixes, I always hem and haw on whether or not it makes sense to keep using the older behavior on old posts, and only use the newer behavior on new posts...

I guess, I don't much like the idea of posts changing in ways that the author didn't account for (for example, I hate that one of my own posts was mangled by the wordfilter at some point after I authored it). I (mostly) lean toward wanting to keep old posts displaying as they did at the time they were authored. For example, in the post I quoted from above, mrb is demonstrating the problem he's describing, as in, if you try to copy-paste his example-sequence, then you'll get the result he's talking about: 79 characters, followed by a space, followed by 41 characters. But, after this fix is applied, if you then tried to copy-paste mrb's example, you'll find that you get 120 contiguous characters, so it'll look (from the perspective of someone reading that post cold) like mrb must have been confused or mistaken when he constructed that example.

I dunno, maybe I'm just overthinking things, but, in case theymos feels that the old behavior is worth preserving, here are two additional diffs that will make sure (at least, in the two places that are important, I think) that old posts won't be affected by this fix:

Code:
--- baseline/Sources/Display.php	2011-02-07 16:45:09.000000000 +0000
+++ modified/Sources/Display.php 2025-02-17 01:11:58.000000000 +0000
@@ -878,24 +878,26 @@
  else
  {
  $memberContext[$message['ID_MEMBER']]['can_view_profile'] = allowedTo('profile_view_any') || ($message['ID_MEMBER'] == $ID_MEMBER && allowedTo('profile_view_own'));
  $memberContext[$message['ID_MEMBER']]['is_topic_starter'] = $message['ID_MEMBER'] == $context['topic_starter_id'];
  }
 
  $memberContext[$message['ID_MEMBER']]['ip'] = $message['posterIP'];
 
  // Do the censor thang.
  censorText($message['body']);
  censorText($message['subject']);
 
+ $context['bbc_use_modern_breaker'] = (int)$message['ID_MSG'] >= 65073000;
+
  // Run BBC interpreter on the message.
  $message['body'] = parse_bbc($message['body'], $message['smileysEnabled'], $message['ID_MSG']);
 
  // Compose the memory eat- I mean message array.
  $output = array(
  'attachment' => loadAttachmentContext($message['ID_MSG']),
  'alternate' => $counter % 2,
  'id' => $message['ID_MSG'],
  'href' => $scripturl . '?topic=' . $topic . '.msg' . $message['ID_MSG'] . '#msg' . $message['ID_MSG'],
  'link' => '<a href="' . $scripturl . '?topic=' . $topic . '.msg' . $message['ID_MSG'] . '#msg' . $message['ID_MSG'] . '">' . $message['subject'] . '</a>',
  'member' => &$memberContext[$message['ID_MEMBER']],
  'icon' => $message['icon'],

Code:
--- baseline/Sources/Profile.php	2013-10-21 19:01:11.000000000 +0000
+++ modified/Sources/Profile.php 2025-02-17 01:12:02.000000000 +0000
@@ -1479,24 +1479,26 @@
  }
 
  // Start counting at the number of the first message displayed.
  $counter = $reverse ? $context['start'] + $maxIndex + 1 : $context['start'];
  $context['posts'] = array();
  $board_ids = array('own' => array(), 'any' => array());
  while ($row = mysql_fetch_assoc($request))
  {
  // Censor....
  censorText($row['body']);
  censorText($row['subject']);
 
+ $context['bbc_use_modern_breaker'] = (int)$row['ID_MSG'] >= 65073000;
+
  // Do the code.
  $row['body'] = parse_bbc($row['body'], $row['smileysEnabled'], $row['ID_MSG']);
 
  // And the array...
  $context['posts'][$counter += $reverse ? -1 : 1] = array(
  'body' => $row['body'],
  'counter' => $counter,
  'category' => array(
  'name' => $row['cname'],
  'id' => $row['ID_CAT']
  ),
LoyceV
Legendary
*
Offline

Activity: 4228
Merit: 23202


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
February 17, 2025, 09:26:05 AM
Merited by Pmalek (3), vapourminer (1), PowerGlove (1)
 #2

I often ran into this, for instance when copying public keys. My solution was to click quote, and copy it from the edit-window. To prevent this, code-tags work fine.
I guess it's one of those little problems I'm so used to, I didn't even realize it can be patched. So it would be good to fix Smiley

¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
[MUSIC] The Ballad of LoyceV
Cyrus
Ninja
Administrator
Legendary
*
Offline

Activity: 4662
Merit: 3959



View Profile
March 27, 2025, 05:47:48 PM
Merited by vapourminer (1), PowerGlove (1)
 #3

Bumping this to confirm that in the account recovery process we bumped into this quite a few times. Thanks for bringing this back up and offering a fix!

joker_josue
Legendary
*
Offline

Activity: 2576
Merit: 7707


**In BTC since 2013**


View Profile WWW
March 27, 2025, 07:59:29 PM
 #4

It turns out it was something I had never noticed before. Maybe because I've never had to select sentences of that length.

It makes perfect sense for this situation to be corrected. And as you mention very well, preventing it from being changed in old posts. What's done is done, let it go.

So a smart solution that would be a good addition to the forum.

▄▄████████████████████▄▄
▄███████▀▀██████▀▀███████▄
█████▀██████████████▀█████
████████▄▄██████▄▄████▀███

██████████████████████████
██▄▄██████████████▄▄██████
██▀▀██████████████████▄▄██
██████▀▀██████████████▀▀██
██████████████████████████
███▄████▀▀██████▀▀████████
█████▄██████████████▄█████
▀███████▄▄██████▄▄███████▀
▀▀████████████████████▀▀
 
 DΞX.fo 
▄▄██████
█████████
██████████
██████████
██████████
█████████
▀▀██████

▄███████
▄██████████
████████████
█████████████
█████████████
|
▄▄█
▄████▀
▄███▀█▄
▄██▀█▄██
█████▀▀█
████████
████████
▀██▄████
▄████▄▄█
▄█████▀███
▄█████▀████▀
█████▀███████
▀██▀█████████
|..BTC......XMR...
..USDT.....LTC...
....Fees  0.8%.....
shahzadafzal
Copper Member
Legendary
*
Offline

Activity: 2366
Merit: 3623



View Profile
March 27, 2025, 08:37:05 PM
 #5

I dunno, maybe I'm just overthinking things, but, in case theymos feels that the old behavior is worth preserving, here are two additional diffs that will make sure (at least, in the two places that are important, I think) that old posts won't be affected by this fix:

Well, if you ask me, that's a bug, and you're fixing it. There shouldn't be any issue implementing this change, and I think it should be fixed in the old posts too. After all, you're just adjusting how it behaves on the front end without affecting the original posts.

On the other hand I guess that's why SMF have [ code ][/ code ] tag for such things.

░░░░▄▄████████████▄
░▄████████████████▀
▄████████████████▀▄█▄
▄███████▀▀░░▄███▀▄████▄
▄██████▀░░░▄███▀░▀██████▄
██████▀░░▄████▄░░░▀██████
██████░░▀▀▀▀░▄▄▄▄░░██████
██████▄░░░▀████▀░░▄██████
▀██████▄░▄███▀░░░▄██████▀
▀████▀▄████░░▄▄███████▀
▀█▀▄████████████████▀
▄████████████████▀░
▀████████████▀▀░░░░
 
 CCECASH 
logfiles
Copper Member
Legendary
*
Offline

Activity: 2898
Merit: 2486



View Profile WWW
March 31, 2025, 12:16:50 AM
Merited by vapourminer (1)
 #6

Hey @PowerGlove

Sorry if this is dumb me asking about here since the cases might not be related. Sometimes there's a way long links get broken, especially those of transaction ID or spreadsheets

And example is this text of mine(I don't want to quote it because it will all fit in)

I just wanted to inform you that 150 USDT has been sent to the address ( TXID ---> https://tronscan.org/#/transaction/3beb8505a451a92a6470eabbf3b67afb4ae59d19a6c0cbea879210b3550a3e9a).

Why doesn't the whole link just drop into the next line if it doesn't fit in the line on top?

LoyceV
Legendary
*
Offline

Activity: 4228
Merit: 23202


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
March 31, 2025, 08:19:42 AM
 #7

I just wanted to inform you that 150 USDT has been sent to the address ( TXID ---> https://tronscan.org/#/transaction/3beb8505a451a92a6470eabbf3b67afb4ae59d19a6c0cbea879210b3550a3e9a).
One fix would be to manually type URL tags, like 3beb8505a451a92a6470eabbf3b67afb4ae59d19a6c0cbea879210b3550a3e9a or tronscan.org.

Quote
Why doesn't the whole link just drop into the next line if it doesn't fit in the line on top?
If that's what you want, just type Enter before the link.

¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
[MUSIC] The Ballad of LoyceV
PowerGlove (OP)
Hero Member
*****
hacker
Offline

Activity: 711
Merit: 7437



View Profile
November 14, 2026, 02:30:41 PM
Merited by vapourminer (1), NotFuzzyWarm (1), suchmoon (1)
 #8

Hey @PowerGlove
Hey. (I'm sorry I ignored your question for so long; probably I meant to reply to you at some point but then just forgot.) Cheesy

Why doesn't the whole link just drop into the next line if it doesn't fit in the line on top?
I'm not the best person to ask. I think what you're noticing is more-or-less the ordinary behavior of your browser trying to prevent overflow by breaking text at so-called soft wrap opportunities, but I suspect that someone who's more into web development will be able to give you a better answer than that. I'm on Tor Browser, and if I hit Ctrl+Shift+C to pick an element off the page and inspect it, and I use that to add something like style="white-space: nowrap;" to your example link, then I can get it to behave in the way that you're asking for (that is, when I horizontally contract my browser window until the example link drops down to the next line, it does so all-at-once instead of in chunks). I think that in "CSS4" it would be more correct to use style="text-wrap-mode: nowrap;". I mean, I realize that none of that really helps you, but, yeah, like I said, I'm not the right guy to hit up with web development questions (it's not a field of work that I enjoy or that I spend much time on [1]).

[1] Before anyone points out that I could just climb on the ability-augmentation bandwagon and use AI to fill out my weak areas: Erm, no. I'm very probably the most anti-AI person you could ever meet, and that's putting it mildly. If someone offered me a big bag of soiled underpants or an unlimited-usage lifetime subscription to some kind of super-secret invitation-only SOTA^6 400-IQ AGI running on a quantum computer of extraterrestrial origin that wouldn't be available to the public for another decade or two, and would at that point cost something like $500k/month to access... I would yawn, scratch my balls, then choose the bag of old undies and be on my way. I don't expect anyone here to understand that decision, but, yeah... I'm diametrically opposed to AI. If I could surgically remove it from the world, jump up and down on it for a bit, and then yeet it into the sun, I would.



Bumping this because I see that this issue has come up again.

@theymos: I tried going through my old PMs to see if I could find where we left things off with this one... I ran out of patience after a few pages of scanning combined with a few likely-seeming Ctrl+F phrases (I was pretty certain that "SMTP" would do it, because I recall its RFC coming up in the conversation, but, apparently not). IIRC, you had a problem with how this patch would affect HTML-to-plaintext conversions. That is, the three existing "breakers" all leave behind a space after being run through strip_tags(), which you felt was behavior that needed to be preserved in a few cases (I think). I may have suggested this at the time, though I don't think I did, but, couldn't you handle those concerns on your end with a string substitution? As in, find the strip_tags() call-sites where you think it's important for there to be remnant word-breaking spaces, and do something like: strip_tags($) -> strip_tags(str_replace('<wbr />', ' ', $)). Anyway, that's just a thought. It would be cool to get this issue fixed, IMHO.
shahzadafzal
Copper Member
Legendary
*
Offline

Activity: 2366
Merit: 3623



View Profile
November 20, 2026, 11:24:21 AM
Merited by vapourminer (1)
 #9

Bumping this because I see that this issue has come up again.

Just a question: Why doesn't SMF apply the same rule to the profile page? Smiley

https://bitcointalk.org/index.php?action=profile;u=976210

Or is it because that particular text box is intended for Bitcoin addresses? Was it added as a custom field specifically for Bitcointalk, rather than being part of the original SMF functionality?

Either way, looking at that profile, I can see why the spacing issue exists.

Good work, PowerGlove! Let's see if theymos considers this a necessary update.

░░░░▄▄████████████▄
░▄████████████████▀
▄████████████████▀▄█▄
▄███████▀▀░░▄███▀▄████▄
▄██████▀░░░▄███▀░▀██████▄
██████▀░░▄████▄░░░▀██████
██████░░▀▀▀▀░▄▄▄▄░░██████
██████▄░░░▀████▀░░▄██████
▀██████▄░▄███▀░░░▄██████▀
▀████▀▄████░░▄▄███████▀
▀█▀▄████████████████▀
▄████████████████▀░
▀████████████▀▀░░░░
 
 CCECASH 
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!