Bitcoin Forum

Btw, I have been looking at Ogg Opus which you were intimately involved in, and kudos on Opus! But it seems one major error was made on the container format Ogg in that it doesn't include markers so that one can't jump forward in a stream efficiently over a bounded bandwidth source such as the internet. I presume the container format was not your responsibility.

Any way, I mention that because I notice Timothy B. Terriberry was your colleague on Opus and I believe he also has a connection to Mozilla and the RUST language development. This caused me to remember that I had been critical of RUST several years ago when it was conceived, because I had some complaint about the way it types invariants. I will try to locate my former criticism and add it to this post.

So I guess it makes sense you would want to experiment with RUST. I am not sure if I would still agree with my former criticism, so I will try to locate it and see if it still makes sense now.

Btw, I applaud the effort to develop zk-snarks for smart contracts. I believe it is absolutely necessary per the revelation in the "cut & choose" thread which if you think about actually has implications for any scripting on a block chain.

The container is very efficiently seek-able over the network, in fact. But your implementation must be sufficiently intelligent.

Here are some benchmarks from the opusfile library, On a 25 hour long variable bitrate audio recording performing 1000 jumps to exact sample positions with no caching:

Total seek operations: 1873 (1.873 per exact seek, 4 maximum).

So an average of less than two operations, and in the worst case 4-- meaning even with a 100ms RTT a worse case seek will not cause a noticeable delay. (and obviously, if it cached, it would be much better.) (In other words, these figures involve absolutely no caching or shared state between the 1000 trials, they're completely and totally independent... if your application only ever did a single seeking operation it would perform an expected 1.873 random reads to do it).

This test is part of the opusfile tests directory, you can perform it yourself on your own files. Even if you constructed a malicious file that looked nothing like actual VBR audio the worst it can do is bisection-- so log() operations-- (I believe the method we use can formally be proven to never perform more than 2x the number of probes as plain bisection on any input no matter how crazy).  It turns out that over large timescales VBR isn't very variable, the variations average out and once your search space is small enough that they don't you've already pretty much landed at the solution.

And this requires no indexes which require the file be written in multiple passes or that it only be seekable when it's "finished"-- a live stream is seekable while it's being recorded. You can truncate or even extract a chunk of the middle of a stream with basically no more complexity that range requests (and putting the header at the front, if you're extracting from the middle), and the resulting output is perfectly seek-able too. Corruption to the file will not break its seek-ability (except, perhaps, for specifically malicious corruption that also fixes up a checksum). And it does this with <1% overhead.

I think the seeking performance, given a good seeking implementation, is pretty good, and it's often more efficient than other less flexible containers even when they have indexes-- because to keep their indexes reasonably sized they're not high resolution-- and getting them requires seeking to the end or a rewrite of the file. To the extent that an average near 2 is less good than 1 might be with a perfect index, that is a cost for the streaming functionality, and I think a reasonable one.

I didn't design the container, but if I did-- I might have only added an additional back-reference or two at each page; I wouldn't change how seeking works.

Tim hasn't really had anything to do with Rust-- I had some infinitesimally small influence on the language (lobbied successfully for overflow of ordinary signed types to be defined as an error). Rust has some nice properties for cryptocurrency infrastructure: in particular it has no overhead, deterministic operation, with high levels of safety enforced at compile time. These things matter for decentralized cryptocurrency, since speed is also a security consideration.

I am not an expert on streaming media and have just begun my research, but it seems to me that your quoted benchmark is assuming many seeks will be done on the stream. But there are cases where the user wants to only skip once or twice into a song as they are sampling the music for the first time, which is my use case. For that case, the lack of an index is afaics horrific because there will be the latency of some roundtrips required to optimally locate the seek position (by a bisection sampling method) and wasted bandwidth as well.

There are many complaints about the lack of an index (https://www.google.com/search?q=ogg+seek+lack+of+index) found on Google search. In particular I note the "Random Access" section of a list of complaints (http://hardwarebug.org/2010/03/03/ogg-objections/) about the Ogg container design.


You apparently assume your container is only going to be used for live streams.

So you are saying Ogg is not designed as an optimal archive format. You could have instead made the index optional.


Hey the resolution could be a configurable parameter so programmers can decide the tradeoff that is ideal for their application. Since when should you decide for them.


Seems you are excluding use cases.

I am getting the idea now after several times interacting with you, that you are clearly better at math than I am but I am better designer than you. You seem to pigeon-hole often. Perhaps it is the heads-down quality that is required to have the patience to learn all that math?

I recognize your intellect and attention to detail, but you seem to also be inflexible which is not the trait of the best software designers I've known in my life. I am guessing that maybe you are strongly German cultured and need everything neatly ordered in your own space (http://esr.ibiblio.org/?p=4901). Note I have some German ancestry (and it shows sometimes in my perfectionism at times), but I also am a mix breed of French, Celtic, and Cherokee native. I think this makes me more creative/flexible than you. Not that I often think about comparing ourselves, just at times like this where you disrespect others (https://bitcointalk.org/index.php?topic=1219023.msg14025849#msg14025849) (and then somehow expect they would respect you  ???).

Perhaps you were not intending to disrespect me in this case, but I think the past track record (see link above) is what leads to this tension which is ready to blow at any time. Also I holding you to a higher standard w.r.t. to this crap that you seem to be foisting with Segregated Witness, because you hold everyone else to such a higher standard and even threatening them of being scammers without sufficient proof (again see my link above). Perhaps I should realize this is just the German trait and brush it off my shoulder. I think the less we interact the better. Thanks for the reply and I wish your orderly orderness didn't rub me the wrong way. Sorry I am about as compatible with a German as I am with shooting myself in the head. I love freedom which means stop stomping on others. I am a sheepdog which means I will fight those who I perceive are oppressing freedom of expression. Hey I understand what it is like to deal with trolls and in that case I would support the swift action since trolls only aim to disrupt, but I wasn't trolling you.


I know I read something about Rust and one of your two names mentioned, but I forgot the specifics and couldn't locate it again readily with a Google search (at least not on Tim's name).

Okay I could understand the incomplete typing of Rust (as compared to say for example Haskell or Scala) would not be the priority when wanting to get as close as possible to the metal while having some higher level functionality not provided by C.

Any way, as I said, the specifics of my former criticisms have been mostly forgotten by myself.  I would need to refresh my memory. I think the flaw was w.r.t. to declaring the invariants for class members in the class methods if I am not mistaken (but that is very vague so I might be recollecting incorrectly).

No, it is assuming the user will make exactly one seek. Otherwise, it would assume caching.  The expected accesses given are what you would get for a user that makes only a single seek.

This is also not a free choice. In existing formats with optional indexes _no_ commonly used software is able to correctly seek when the "optional" index is not present. Many have no support for that case at all at all, and what has support in theory doesn't work due to inadequate testing. Doing so would also increase the overhead for the format by 20% or so (as in if the overhead for a file was 100k, it ends up being 120k). As mentioned, accurate indexes are not small-- and many things compromise by just not providing accurate indexes; which then leaves applications linearly scanning or not permitting sample accurate seeking.

What use case is excluded? Moreover-- what error is there in having a streaming container that is actually good at streaming? People can use other containers if they need other properties.  (Though ironically, the best supported non-streaming container that is widely used with Opus-- MKV-- which has 'optional' indexes, on average requires transferring more data to seek a single time than Opus in Ogg does; due to the need to transfer the whole index to seek only once). The height of flexibility is acceptance that not every goal can be optimally satisfied in a single design.

At the end of the day there are always trade-offs. The design of Ogg makes seekable streaming encoding possible, in alternatives with indexes that isn't possible at all. As a result the seeking is less efficient but with a good implementation only _slightly_-- e.g. if you seek 0 to 3 or so times, it'll still transfer less data than an format with an index. Many people are surprised at how efficient it is because they don't realize how much better secant/brent's method of root-finding is over bisection. I think the trade-off is a very good one, but if for some reason it doesn't work for your applications you're free to use another container. MKV support for Opus is well specified.  Someone caring about different things than you do does not make them bad or stupid, I hope you realize that, especially when their preference hardly hurts yours. I kinda feel like you're missing out on an opportunity to be amazed at how efficient root finding can be and learn something new and surprising to you. :(

I have no clue what you're going on about here. I responded politely to your message here-- are you referring to my non-amusement to your wall of insults, allegations, and personal attacks that you cross-posted to several threads; when you couldn't find your own post when it had been moved to offtopic?   I apologize if my abrasive response makes you feel threatened enough to need to expound on your best racist theories; but I have had several of our best posters in the past tell me they stopped using Bitcoin Talk because of harassment by you in particular, and its certainly a situation I've felt myself. You may not intend to troll; but your radical jumps into insults and attacks combined with your persistence and the stream of consciousness diving into incoherent technobabble arguments which you sometimes present is effectively indistinguishable from trolling. I and others have expended a lot of time trying to encourage better behavior under your now-banned past identities, without success. All that is left is to say that if you are unable to behave yourself and not act like a troll, you will be banned. Otherwise you won't be. The choice is yours.
 

Late reply because I was constructing a wooden cage to hold a ceiling fan since the house I rent is so poorly designed that there are no wall nor ceiling studs capable of supporting a ceiling fan. So I lost most of the day being a carpenter without power tools.


I can't find the details for this benchmark any where on the Opus website. I presume you are expecting that I would have to download, install, and build from source and then study the source code to understand what the benchmark is actually measuring. That seems to be an inappropriate level of work for an interested person to do when they simply want to get a reasonable first-pass analysis of the performance issues.

As you know, the devil is in the details. I don't know for example if the benchmark is using some heuristics to reduce the error for estimation of the file position for seeks given it has been given 1000 trials. Also where you wrote "jumps to exact sample positions", I don't know if you mean the jumps are to exact positions in time and thus estimations of file position. Also I don't know the frame size that was employed (i.e. I know Opus goes as low as 20ms but this increases the overhead) and whether that is a significant factor. I think I read that an entire page much be downloaded before the checksum can be computed, to be sure one doesn't confuse garbled data with a frame header.

I presume the error in estimation would increase if the sound content was highly variable in complexity and thus the VBR encoding would generate highly variable file positions relative to desired seek position in time. I am wondering if the 4 seeks worst case would hold. What if it climbed to 8 seeks!

I didn't check how large the page size is, so I don't know if bandwidth wasted is a significant factor with erroneous seeks in the file position.

In any case, one certain error in your analysis appears to be that you presume the internet can be modelled with a mean latency (e.g. 100ms). The internet I presume has hiccups and I presume the latency is Gaussian distributed. Thus 1 or 2 extra round-trips (e.g. 100ms each direction ideally, and worse from the Davao, Mindanao where I am) could be the difference between an unnoticeable delay of for example 500ms or a noticeable delay of 1+ seconds. And the occasional 4+ seeks, could mean up to seconds of delay. What may be more compelling is the extra load on the server, the hard disks thrashing, and also the latency of the server on each seek.

Perhaps this has already all been studied exhaustively, but unfortunately I can't find that data with Google, so it appears to me to be another set of risks. Adopters have to factor in risks in their assessment.


This was a similar argument made by the IETF group working on the Websocket framing protocol when I argued they should make the framing orthogonal to the data layer. I quit because they were not going to change their mind and I presume they did not adopt my proposal.

---

---

I think it is not a valid argument because many implementations may not correctly implement the more obtuse seeking without an index. Thus we get the worst of both worlds.

I just don't buy this notion that we should dumb down and make inferior standards, because people are lazy to write and test software properly. Geez did Vint Cerf follow that theology when he design TCP/IP? Did we conflate all the network layers and collapse them into one layer because the OSI model is complex. Hell no! The End-to-End Principle is fundamental!


I assume the 20% estimate is only for when the optional index is present. So it is presumed someone would use an index only when that 20% was justified by their use case. Again I argue you should not remove degrees-of-freedom and hinder the optimization of use cases which you did not envision because no group or person is omniscient.

And how is not having the index any worse than not allowing an index. I fail to see the logic. Seems you are arguing that the receiving end will expect indexes and not be prepared for the case where indexes are not present. But that is a bug in the receiving end's software then. And in that case, there is no assurance that software would have done the index-less seeking more efficiently for the status quo of not allowing an index. None of this makes sense to me.

Also I don't understand how you calculate 20% increase in file size for adding an index. For example, lets take an average 180 second song consuming roughly 5MB for VBR encoding. Let's assume my users are satisfied with seeking in 1 second increments, so that means means I need at most 180 of 22-bit indices, so that is only 495 bytes which is only a 0.01% increase! On top of that I could even compress those 22-bit indices into relative offsets if I want to shrink it by roughly 75% to 0.0025%.


I have alluded to scenarios in this post. Permute them. And there are use cases that neither of us are aware of. We are not omniscient. We should not top-down remove degrees-of-freedom. Some of the people designing Web standards these days are doing it wrong and are not of the same pedigree as Vint Cerf. I butted heads with them in the past (including Ian Hickson!), and I am tired of arguing with you guys. Just do what ever you want, I am going to route around the failure.


Because it is very difficult to get new standards widely promulgated. The cost of not having Opus supported well for use cases. I will probably end up using Opus and having to roll my own client side instead of using HTML5 (which means my stuff will work best only on Android and not in the browser and probably not as well on iOS at least at the start). So I am very thankful for the work you did on Opus if I go that route. But I just think you should acknowledge to yourself that you are really awesome at your core competency, but you math guys need to allow guys like myself to participate in the standardization process for high-level issues. I have worked for example on what is now Corel Painter (the natural media paint tool not Corel Paint). Meaning I have dealt with the masses (Painter-J shipped a million units in the 1990s for example).

Here is an example of where I promised Mozilla they were shooting themselves in the foot (https://bugzilla.mozilla.org/show_bug.cgi?id=588292#c38) and they obstinately refused to listen, even after dozens of developers complained as I promised Mozilla would happen.


Perhaps their index is very high resolution, very lengthy stream, and/or there is no option to break the index up so not the entire index is put at the start of the file (all of which could be fixed with a better protocol design and specification). I provided my computation above for a 495 byte index.

Hey I have absolutely no experience at all with streaming. I am doing this all in my head for the first time. I think that exemplifies that I am an experienced coder and designer.


My reply is here (https://bitcointalk.org/index.php?topic=1219023.msg14032362#msg14032362).

Gregory I worked for Tom Hedges on Painter and he could memorize a page of the phone book by looking at it for few moments. His IQ was off the charts. He would send me 1 sentence directives and it was up to me to figure out what was all reachable from that one sentence.

What may seem incoherent to you is perhaps because you are not investing the time to follow links and the discussion that has taken place. It is inane to suggest I should re-type everything I already typed over and over again just so that no one has to follow a link (https://bitcointalk.org/index.php?topic=1342065.msg14027714#msg14027714).

Actually Gregory, several times I have felt I respect you and admire your work and even presentations. Obviously I do, because I want to use some of your work. And I've learned important things from you. I have already explained what seems to rub me the wrong way (https://bitcointalk.org/index.php?topic=1219023.msg14032362#msg14032362). I don't know how to fix that, other than to stop interacting.

Btw I think I did actually figure out how to remove the proof-of-square in CCT (but perhaps I have a math error but I have looked it several times and can't find an error but it just seems too simple and I don't know how you guys could have missed it, so there must be an error). I will release the white paper where I had combined CCT with CN rings back in June/July. But I want to reorganize the paper a bit based on some insights I gained from the discussions with Shen-noether.

As for the politics, never mind (insoluble). People will think what ever they think. I have my justifications and reasons, but it isn't worth trying to convince others of my side. It is time for head in the sand, coding. Thanks. Talk is cheap as you know.

I wanted to add something about the "rascist" allegation. I once attended an all negro elementary school in Baton Rouge, Louisiana, USA where my sister and I were the only non-negro students in the school. The kids had never felt fine hair, so they were constantly rubbing my head from behind (in the halls, etc) when I wasn't looking. I learned that negros have oily hands.

I am far from rascist, as I live in the Philippines and have a brown skinned gf. I have kids who are half-filipino.

But what is more interesting to me is your generation is so ideologically preoccupied. Perhaps you believe in the lie of anthropogenic global warming as well. I hate to bust your bubble but it was a propaganda constructed to put in a place a global governance and taxation system. The scientists have confirmed we are headed into a Mini Ice Age and our global temperature is modulated by the sun, nothing else. Period.

You are also similarly naive about who controls Bitcoin (https://bitcointalk.org/index.php?topic=1378112.msg14035772#msg14035772) and what their objectives are for global control. You are just a pawn Gregory. I hope you wake up and use your talent to fight for freedom, which means don't stomp on others and fall victim to incorrect perceptions (https://bitcointalk.org/index.php?topic=1219023.msg14032995#msg14032995) which you want to enforce on others.

I wish you the best.

No wonder why noone takes you seriously.

Feel sorry for the kids. Half-Filipino......half-petrified.  :'(

This is just getting better and better! Please, more! Levels of delusion and insanity starting to reach terminal velocity...

Amazing.

I didn't take him seriously before but that quote is priceless.

Especially because after I embarrassed Gmaxwell upthread on the technical points, now I am going to embarrass you guys with some facts (but I realize your AGW religion ignores facts):

https://bitcointalk.org/index.php?topic=1091272.msg14688442#msg14688442

https://bitcointalk.org/index.php?topic=1091272.msg14660362#msg14660362

Here is a few morsels of technical kryptonite for the "meme image self-masturbation" trolls[ducks]cock-suckers to gag on:

https://bitcointalk.org/index.php?topic=1459846.msg14766916#msg14766916

So they want to play hardball... okay I know that game too...

Whose cock are you sucking today boiz? Gmax? Bitalik? Sincereo? Duckefeller? Cat got your tongue or mouth too full of yummy finger-licking-good semen?

The low-life anonymous scum that is here on BCT:


What kind of man talks about another man's kids on a public forum hiding behind anonymity. That isn't a man. I dare him to say that to my face. You run out of ammunition to attack my reputation with, so you involve my kids who have nothing to do with anything that is going on here on this putrefied forum.

You brought your kids into this forum, not me.

Reputation? How does that look from your side of the screen, exactly?

Im not going to fuel your narcissistic self-important needs here, so yeah, anytime you come to Europe, look me up. Happy to do face to face stuff. No problem.

Pretending you are dumb, isn't a very attractive nor convincing retort.

TPTB_need_war mentioned interracial children amongst other points as evidence to refute gmaxwell's ludicrous allegation of racism.

Attacking a the reputation of a man's children (when those children aren't even here to defend themselves) from behind an anonymous internet account is abusive and I dare say maybe prosecutable in the UK and/or the USA. Only a very dumb person would assume that your words wouldn't be also hurtful and damaging to the children as well.

But it is not prosecution that interests a real man. Rather it is that if you were a man, you would not say that even once without being in front of the accused's face because you should understand that a man's family is his private personal space that is not fair game for internet hyperbole. TPTB_need_war is not a public figure vying for political office, and besides even if he was, you are speaking to him directly and personally, not as some political discussion with others. You directly attack his personal space from the cowardly distance of anonymity and the internet wire. Sorry there is no retort that you can make which will restore your reputation from the sleezeball that you have made yourself. The only two things you could do to redeem yourself is a) apologize and admit it was cowardly and unmanly, b) travel on your own expensive to his location to say to his face in a place where there is no law and no cameras to protect you from his reaction.


So brave, so europe0n (https://bitcointalk.org/index.php?topic=1316268.msg14767690#msg14767690).  ::)


Yeah how does it look (https://bitcointalk.org/index.php?topic=1459846.msg14776165#msg14776165).

Or how is it in reality (https://bitcointalk.org/index.php?topic=1438301.msg14757751#msg14757751).

Enjoy the utility of your aggrandizing reply in this echo chamber of technical errors and non-accomplishment. BitCON with 70% (soon 98%) of the mining controlled from a few Chinese mining farms.  ::)

Good job boiz!  :'(

You make your children read these posts, as well as listen to your version of your world order?? Like i originally said.......absolutely petrifying.    :'(


  

You did it to yourselves (https://bitcointalk.org/index.php?topic=1316268.0). Now you will reap what you have sown.

I am an American who doesn't share your looney European Marxism (https://bitcointalk.org/index.php?topic=1465136.msg14789764#msg14789764). Last time it was a million in the gas chambers. Let's see how it goes this round.

Shut up and get back to work on building your copy-leftist clusterfuck (https://bitcointalk.org/index.php?topic=1459846.msg14781570#msg14781570).

I don't associate with scum like you. I compete and overcome. Bye. Unless that is you want to say those words about my kids to my face. Otherwise we have nothing more to discuss. Enjoy your life.

What kind of reaction are you eluding to, exactly?

I suggest you ask him and to his face.

If you were a real man, you'd not say such things about another man's children.

The discussion is finished. You are wasting my time. I do hope you have more important matters to attend to.

I suggest you keep your trap shut, when it involves intentions you are not capable of living up to.

Im not wasting your time. This is what you do. You choose to have to pointless arguments with people over and over again. Pathetic really, for an obviously intelligent man. Recently you have become very abusive. Even sending threatening PMs to people, and now you are suggesting a "reaction" out the sight of "law and cameras". You seem to be on a trajectory that will end in tears TPTB. Your precious children are gonna be so proud of warrior dad.  :'(



 

It is quite pervasive:

Im just highlighting the absurdity of your hypocrisy. Whilst claiming that some authority somewhere in the world could find "Half Philipino, half petrified" prosecutable, there you are posting a veiled physical threat in the same post ...... "in a place where there is no law and no cameras to protect you from his reaction."
There you are calling this forum "putrified", and claiming your time is wasted here, but still have the time to post pointless, 'You are: scum, sleazy, sleazeball, blah blah blah posts. So the question could be: why do persist in harassing this forum? How many account bans do you want, before you understand that the putrified nature of any forum, comes from abusive, divisive, and threatening individuals like you.

Quoting that because I don't want it to be lost in the future when historians and the press are documenting the man who built an altcoin that displaced Bitcoin.

Because to you, being abusive, divisive, and threatening, makes you feel important, right? 

Whens the whitepaper coming for CertifiedBSCCcoin, then?