開源補助只評三成，預測市場怎麼分錢？

0:00開場：二元、標量、多標量、條件市場

0:00謝謝介紹，Shawn Woo。我也是因為他才會在這裡。分配資金的過程裡，他是我們的評審之一，做得很好。現場有多少人當過資助者，或評過專案？有人當過資助者嗎？一、二。多少人評過專案？幫資助者當過評審。一。多少人拿過 grant 或其他補助？一、二、三、四。好，稍多一點。所以我覺得組合還不錯。Thanks for the intro, Shawn Woo. He's also the reason I'm here. He was one of our evaluators during our process to distribute funding, and he was very good at his work. So how many here have, like, either been a funder or evaluated projects? Anyone has been, like, a funder? One, two. How many have evaluated projects? You've been an evaluator for a funder. One. How many have received funding from, like, grants and other things? One, two, three, four. Okay, slightly more. So, okay, we have a decent mix, I think.

0:47多少人知道預測市場是什麼、基本怎麼運作？好，大概五位。那我假設大家知道得有限。最基本的形式就是大家熟的：Trump 贏還是 Harris 贏。贏的變成 100，輸的變成 0。這是我們看到最基本的預測市場。等一下結束前我會出考題，所以請專心，考試要來了。第二種叫標量市場，這些市場不是結算成 0 或 1，而是結算成百分比。如果我們預測民主黨會拿 45% 席次，實際拿了 50%，你的利潤只有 5 美分，因為從 45% 走到 50%。How many of you know what prediction markets are and the basics of how they work? Okay, so around five. So I'm going to assume, like, limited knowledge. Yeah. So the most basic form is what we are familiar with, which is, like, will Trump win or Harris? And whichever wins becomes 100, and whichever loses becomes zero. So that's the most basic form of prediction markets that we see. Slightly. And I'm going to ask an exam at the end of this, so, like, all of you pay attention because there's exam time coming up. Like, the second form is what's called scalar markets, and these markets don't resolve to zero or one. They resolve to the percentage. So if we predict that Democrats will win 45% of the seats, but then they actually end up winning 50%, your profit is only 5 cents because from 45% it goes to 50%.

1:39你實際賺到的就這麼多。不像前面那種是 0 或 1，這裡是一個分布。45 變成 50，或 55、54 變成 50。這是預測市場的第二種。這是多標量市場，對今天在台灣的我們比較用得上。這裡你要預測各黨會拿多少席次比例。你可能說 EPP 會拿 188 席，交易者賺多少，取決於你對得有多準。你猜得非常準、而市場價格差很遠，你就賺比較多。這就是多標量。So that's all that you actually end up making. So unlike here, it's not zero or one. Here, it's a distribution. 45 becomes 50, or 55, 54 becomes 50. So that's kind of the second form of prediction markets. This is multi-scalar markets, which is more applicable to some of us today here in Taiwan. Like here, you're trying to predict how much, what is the percentage of seats that each party will win. So here you might say that EPP will win 188 seats, and your amount that you earn as a trader is based on how right you are. How much did you guess the percentage? If you guess it very correctly, then you get, like, and the market price is very wrong, then you get more. So that's multi-scalar.

2:24現在進入真正有意思的部分：條件市場。你要決定我們黨該推誰去選。提名人該是 Trump、Harris，還是 Biden？這個問題很難答。我覺得民主黨輸，就是因為這裡做了錯的決定。條件市場幫得上忙：交易者會給資訊，例如 Biden 是提名人，民主黨有 30% 勝率；Harris 是提名人，民主黨大概 25% 勝率。用條件市場，你其實能拿到該採取什麼行動、該做什麼決定的資訊。Now we start getting into the really interesting parts, which is conditional markets, where you're trying to decide who should the nominee be for my party to fight the election. Should my nominee be Trump, or should my nominee be Harris, or should it be Biden? So this is a very difficult question to answer. Like, I think the Democrats lost because they made the wrong decision over here. So this is where conditional markets help, where the market, like people, like traders, are giving their information about if Biden is the nominee, Democrats have a 30% chance of winning. If Harris is the nominee, Democrats have, like, a 25% chance of winning. So, like, with conditional markets, you can actually get information about what action you should take, about what decision you should make.

3:14以太坊依賴圖太大，條件市場用來分補助

3:14現在接到補助。我們想看一個 repo 值多少。基本上會問：這個 repo 會被評嗎？Shawn Woo 是評審，但時間有限，他沒辦法看每一個專案。我們說：會不會被評？如果被評，分數會是多少？所有交易者都在預測專案分數，真正被評的只有一部分。那才決定他們的損益，也才給他們誘因。這就是用到補助上的方式。跑市場有很多好處，等一下會講。最大的好處是這個。這是整張以太坊依賴圖。And this is now where we come to applying it to funding, where here we are trying to see, like, what would a repo be worth. So we basically ask something like, will the repo be evaluated? Shawn Woo was an evaluator, but he has limited time. He can't look at every project and evaluate it. So we are saying that will it be evaluated, and if it is evaluated, what score will it receive? So, like, all of the traders are trying to predict the score of a project, and only some projects will actually get evaluated. And that's what determines their profit and loss, and that's what gives them an incentive to do it. So that's kind of how you can apply it to funding. There are many benefits to running a market, which I'll get into soon. Yeah, the biggest benefit is this. This is like the entire Ethereum dependency graph.

4:02你可以看到圖上大概有 10,000 條邊，還有 4,000 個開源 repo。沒有評審能把這些都看完再給一個數字。誰都做不到。所以我們可以跑條件市場：如果評審評了，分數會是多少？真正拿到評分的只有幾個，但還是可以把錢分到整張圖。這其實是 Vitalik 出過的題：有沒有可能你只給我一把公鑰，我把錢放進去，它就自動分到整張圖？這就是一種試法。架構大概是這樣。好。There are, as you can see, like 10,000 different edges, or lines, in the graph, and there are 4,000 different open-source repos. Like, no evaluator is able to look at all of these and then actually give, like, a number. It's, like, impossible for anyone to do that. So this is where we can run conditional markets, where we can run a market saying, if an evaluator evaluated, what score would it receive? And then we can only actually get evaluations for a few of them, but still put money and distribute to all of this. This was actually a prompt which Vitalik had given, which is: Is it possible that you can just give me one public key, I put money in it, and it automatically distributes into the entire graph? So this is kind of a way to try making that happen. This is kind of the architecture as it works. Cool.

4:49第一層、第二層、第三層。看例子比較清楚，我就不細講。快速複習，因為考試要來了，各位要開始答題。基本版我叫二元市場。二元市場只有兩個選項。一個變成 100，另一個變成 0。這是二元。標量市場也只有兩個選項，但不是 100 或 0，可以是 60 跟 40、50-50，可以結算成任何數，不必是 100 跟 0。標量市場。多標量市場則超過兩個選項。再來是條件市場，不一定要全部結算。So the first level, second level, third level. It's better to see the example, so I'm not gonna, like, run into this. But, like, quickly, revision time because exam is next, so all of you have to start answering questions. So yeah, you have the basic version, which I call binary markets. Binary markets are only two options. One option becomes 100, the other option becomes zero. That's binary. Scalar markets, where there are only two options. But it's not 100 or zero. It's like it could be 60 and 40, 50-50. It can resolve to anything. It doesn't have to be 100 and zero. Scalar markets. Multi-scalar markets, where there are more than two options. So that's kind of the... And then you have conditional markets, which doesn't have to be resolved.

5:36記住這個框架，現在開始考試。這是 Deep Funding 1，我們有 98 個 repo，問大家每個 repo 有多重要。你可以看到，市場說 go-ethereum 值 4.9%，Solidity 值 4.67%，Consensus Specs 4.3。這些全部要加總成 1，全部加總成 100。所以你放 100 美元進這個分布，4.9 美元會到 go-ethereum。市場就是這樣運作。第一題：這是多標量、標量，還是二元市場？可以直接喊答案。So that's the kind of—just remember this framework, and now we'll start with exams. So this is Deep Funding 1, where we had 98 repos, and we asked people how important each repo is. So here you can see that, like, go-ethereum, the market said it's worth 4.9%. Solidity is worth 4.67%. Consensus Specs, 4.3. And all of these have to add up to one. All of these add up to 100. So you can put money—if you put $100 into this distribution, $4.9 will go to go-ethereum. So that's kind of how the market works. So question one: is this a multi-scalar market, a scalar market, or a binary market? You can just shout out the answer, yeah.

6:24Deep Funding 1 是類別多標量，98 個都要結算

6:24多標量？對，正確。這是多標量市場。你答對是因為選項很多。第二題難一點。這是條件市場還是類別市場？對，是類別市場，對吧？為什麼？因為一個的補助不依賴另一個。一個的補助不依賴另一個。有些情況會，但這個例子裡沒有。對。答案對了，理由錯了。一個的補助確實依賴另一個，因為全部要加總成 100。Multi-scalar? Yeah, that's correct. It's a multi-scalar market. You're correct over here because there are many options. Second question, this is a little harder. Is this a conditional market or a categorical market? Yeah. It's a categorical market, right? Okay, why? Because the funding of one isn't dependent on the other. The funding of one isn't dependent on the other. Well, in some cases, yeah, but in this example, no. Yeah. So you got the answer right, but the reason wrong. The funding of one is dependent on the other because, like, they all have to add up to 100.

7:10因為加總成 100，一個 repo 拿 10%，其他就只能搶 90%，對吧？某種意義上我們碰到 Arrow 不可能定理。你試著加進新的 repo，就會打亂其他人的權重。這不是條件市場，因為每一個 repo 都要評。所以你是對的：這是類別市場，不是條件市場。我其實問過 Vitalik，98 個 repo 可不可以不用全部拿到答案，因為要評審涵蓋全部太難。他說那製造的問題比解決的還多。多標量市場不能部分結算，每個結果都要結算。所以這是類別多標量市場。第一題就這樣。So because they add up to 100, if one repo is getting 10%, that means the others can only compete for 90%, right? So in a sense, so like we run into Arrow's impossibility theorem. If you try entering a new repo, then it upsets the weights of the others. So this is not a conditional market because you have to evaluate every single repo. So that's why you're right. It is a categorical market. It is not a conditional market. I actually asked Vitalik, is it fine if we don't get answers on all 98 repos because it's super hard to get evaluators to cover all 98? And he said it actually creates more problems than it solves. You can't partially resolve a multi-scalar market. You have to resolve every single outcome. So this is a categorical multi-scalar market. So that's question one.

7:59這裡一個答對，一個答案對、理由錯。好。現在第二題。這叫 repo 的原創性：多少錢該留在這個 repository，多少該傳給它的依賴。例子像 Brave Browser。Brave Browser 建在 Chromium 的 fork 上。要給準確的補助，不該把錢、或百分之百都給 Brave Browser，應該多給 Chromium。Solidity 則很原創，依賴很少，從零做起，所以該把錢留下。這個市場就是在問這個。We got one right answer and one right answer but wrong reason here. Yeah. Yeah. So this is the second question now. So this is like what's called the originality of a repo, which is how much money should stay with the repository versus get passed on to its dependencies. So, like, the example here would be something like Brave Browser. Brave Browser is built on a fork of Chromium. So if you want to give accurate funding, you should actually not give the money to Brave Browser or all 100% of it. You should give more to Chromium. Whereas something like Solidity is very original. It has very few dependencies. It's built from zero, so they should keep all the money. So that's what this market is.

8:46這裡講的是 Supranational BLST，一個超重要的非加密 repo，加密世界大家都依賴它。市場說它 74% 原創。74% 原創的意思是 74% 的錢留在 BLST，26% 傳給依賴。第二題：這是簡單標量、多標量，還是二元市場？還是多標量，對吧？為什麼？因為全部加總成 1。喔不，它們是獨立的。對，它們獨立。例如 up 是 74，down 是 26。Here we are saying, like, Supranational BLST, which is like a super important non-crypto repo, which we all depend on in the crypto world. So here the market is saying that it is 74% original. So 74% original means that 74% of the money stays with BLST, and 26% gets passed on to its dependencies. So yeah, second question: is this a simple scalar, multi-scalar, or binary market? Still multi-scalar, right? Why? Because it all adds up to one. Oh no, it's independent. Yeah, they are independent. Like if up value is 74 and the down value is 26 for that.

9:35原創性是簡單標量條件市場，試點只評了三成

9:35那你要改答案嗎？不改。好，多標量是一個答案。還有人有別的答案嗎？標量。標量？簡單標量還是多標量？簡單標量。對，正確答案是簡單標量。理由是 up 加 down 等於 1。所以每一組都獨立。只有兩個選項，這是主要差別。多標量有很多選項。選項多就是多標量。這是簡單標量，因為只有兩個選項：BLST，74% up、26% down。So, do you want to change your answer? No, no. Okay, multi-scalar is one answer. Anyone else have another answer? Scalar one. Scalar? Yeah, like simple scalar or multi? Simple scalar, yeah. Okay, that is the correct answer: simple scalar. The reason is that up plus down is equal to one. So each of these are independent. There are just two options. That's the main difference. A multi-scalar has many options, right? So when there are many options, then it's a multi-scalar. This is a simple scalar because there are just two options: BLST, 74% up, 26% down.

10:21每個 repo 只有兩個選項，交易者在預測的就是這個。所以這是簡單標量。那它是條件市場還是類別市場？我的意思是：每個 repository 都要結算，還是可以只拿幾個答案、其他不用？我們只需要有限選項，所以是條件。是條件市場。對，也正確。哇，你很強，這題也過了。是條件市場，因為其實不必全部拿到答案。在我們的So there are just two options per repo, which traders are actually, like, trying to predict. So this is a simple scalar. Now, is it a conditional or a categorical market? By which I mean, do you need to resolve each of these repositories, or is it possible to just get a few answers and not get all of the other answers? We only need the limited options, so it's conditional. It's a conditional market. Yeah, that's also correct. Yeah. Wow, you're really like an ace, you know, you've aced this. So it's a conditional market because you don't actually have to get answers for all. In our, in our...

11:12試點裡，大約只有 30% 的 repo 拿到答案。剩下 70%，評審沒時間真正作答。但 30% 有答案，這就決定了所有交易者的損益。好消息是所有 repo 都有了答案，人為資料其實只收了子集，大約 30%。這就是跑這些市場的好處：可以放大。這是比較大的一層，我們想幫 4,677 個依賴給權重。例子是 Vyper。你看到 Titanoboa，Vyper 語言的一個 repository。我們說 vvm 值 30%，eth-account 值 17.42%。pilot, we only got answers for about 30% of the repos. So the remaining 70%, evaluators did not have time to actually get answers on it. But we did get answers for 30% of them, and that's what gave profit and loss to all the traders. So the good news is that we got answers for all the repos, but we actually collected human data for only like a subset, like 30% of it. So that's kind of the advantage of running these markets. You can scale it up. So this is kind of the big level where we were trying to get weights to 4,677 dependencies. And so, like, this is an example of Vyper. So you're seeing Titanoboa, which is a repository of Vyper language. We are saying that something like— vvm is worth 30%. eth-account is worth 17.42%.

12:03這些就是組成這個 repo 的不同依賴。看下一個 repo，只是 Vyper，不是 Titanoboa，你可以看到 CBOR 值大約 0.00005。這樣我們其實可以給整張依賴圖權重，全部 4,677 個。這是多標量、簡單標量，還是二元市場？這邊還沒人答，有人要猜嗎？簡單標量。簡單標量，一個答案。有人有不同答案嗎？So this is like the different dependencies that make up the repo. If you look at the next repo, which is just Vyper, not Titanoboa, there you can see CBOR is worth like 0.00005. So like this, we can actually like— get weights for the entire dependency graph, which is all 4,677. So is this a multi-scalar, simple scalar, or binary market? We've not got any answer from this side, so anyone here want to take a guess? Simple scalar. Simple scalar, one answer. Anyone have a different answer?

12:56依賴權重可條件結算，Zcash 用二元市場做分流

12:56其實答錯了。這不是簡單標量，因為每個值依賴其他值。這是多標量，選項很多。左邊這些 repo 全部要加總成 1。所以跟第一個市場一樣，是多標量，因為選項很多。這是條件市場還是類別？可以再猜。條件還是類別？類別？算部分對，因為如果選了一個 repo，就要評完它所有依賴。但差別是它也是條件，因為不必選每一個 repo。So it is the wrong answer, actually. There's like, because this is not a simple scalar, because each of the values depend on the other. It's a multi-scalar, many options. So here you can see all of, on the left-hand side, you get all of these repos, which have to add up to one. So it's similar to the first market, which is a multi-scalar because there are so many options. Is this a conditional market or a categorical? You can take another guess at it. Conditional or categorical? Categorical? So that's like partially correct, because if we choose a repo, we have to evaluate all of its different dependencies. But the difference is that it's also conditional because we don't have to choose every single repo.

13:42如果選 Titanoboa，就要評完它所有依賴，所以是類別。但我們可以只選 Vyper、不選 Titanoboa，那就變成條件市場。這個例子裡，98 個 repo、4,600 個依賴，我們實際收資料的大概只有 12 個 repo。交易者等於在市場上用 6 倍槓桿交易，因為損益只取決於那 12 個真正收了資料的 repo。那 98 個 repo 以及它們的 3,000 個依賴，我們用他們的答案去分錢，但那些市場其實沒結算。我們只結算這些。這就是例子。So, like, if we choose Titanoboa, then we have to evaluate all of its dependencies, so it is categorical. But we can just choose Vyper and not choose Titanoboa. In that case, it becomes a conditional market. So for a lot of the, like in this case, out of 98 repos and 4,600 dependencies, we only actually collected data for, like, maybe 12 repos. So traders were trading at a 6x leverage in the market because their profit and loss just depended on those 12 repos that we actually collected data for. For the 98 repos and 3,000 dependencies of those 98 repos, we used their answers to distribute money, but we didn't actually resolve those markets. We only resolved these. So that's kind of the example.

14:30這只是我們跑過的大練習的一塊。我們後來發現，預測市場其實可以更廣：任何補助輪次都可以跑一個預測市場。現在有一輪 Zcash retro round，37 個專案申請補助，要評 37 個很難。我們就開了預測市場：這個專案被 Zcash 代幣投票者通過的機率是多少？代幣投票者要投每個專案時，先來看我們的預測市場、看賠率，再做分流。他們會看到，例如 Frontier Compute Zcash 只有 33%。This is like one part of the big exercise we ran, but something we realized is that we can actually, like, use these prediction markets in a more broader way, that if there's any funding round happening, we can just run a prediction market on that funding round. So, like, right now there's a Zcash retro round where 37 projects have applied for funding, and it's very hard to look at 37 projects and evaluate them. So we just started a prediction market saying, what is the odds that this project is going to be approved by the Zcash token voters? So now when token voters have to actually vote on each project, they just go to our prediction market, they see the odds, and then they can actually do triage. They can see that, okay, this project like Frontier Compute Zcash, oh, only 33%.

15:15那我對這個專案多審一眼。或像 Ironwood 是 87%，機率很高。他們就可以校正預期，少花一點時間審查，因為市場已經給了高機率。這些值都會轉換。這是二元還是標量？我聽到有人喊二元，正確，因為專案拿到補助就變成 100，沒拿到就是 0。這是二元市場。好，最後幾張投影片。這是我們拿到的大好處。例如從 Deep Funding，我們看以太坊各客戶端的分布。根據市場。Let me apply some extra scrutiny to this project. Or something like Ironwood is 87%, so that's like a very high odds. So then they can calibrate their expectations, and they can, like, spend less time reviewing it because the market is already assigning a high probability. And all of these values will convert. So, like, is this a binary market or a scalar, or is this a binary or a scalar? I heard someone shout binary, and that's the correct answer because if the project gets funded, it converts to 100. If it doesn't get funded, it's zero. So that's a binary market. So yeah, like this is the last slides now. This is like the big advantage of, like, what we got. So like, for example, from Deep Funding, we looked at what the breakdown of each client distribution is for Ethereum. So according to the market.

16:04評審跟市場對不上：Teku 17%，Octant 更喜歡市場權重

16:04根據評審投票，也根據獲勝模型。Vitalik 看到這份名單時說，喔，Teku 在評審票裡佔很高，大概 17%。我們回去問評審，他們說對，絕對不該是 17%，應該低很多。跑這個市場，你可以找出落差：委員會講這個，市場預測另一個。再回去問委員會：市場跟你們差很多，原因是什麼？這就是一個好處。這是 Octant。Octant 是一個專案。最後一張。Octant 質押 ETH，再把錢分出去。According to the votes of the jurors and according to the winning model. So something which Vitalik noticed when we sent him this list, he said, oh, Teku has a very high amount according to the juror votes. It's like 17%. And then when we went back to the jurors, they said, yeah, it should definitely not be, like, 17%. It should be much less. So by actually running this market, you can identify discrepancies, that the committee is saying this, but the market predicted another value. So let's ask the committee now, hey, the market and your values are very off. What is the reason for that? So that's kind of an advantage. This was in Octant. Octant is like a project. This is the last slide. This is like the—Octant stakes ETH, and they distribute money.

16:49我們跑了誰會拿多少錢的預測市場，Octant 主辦方和其他人也說他們比較喜歡預測市場的結果。我們把這張貼到 Twitter，大家都說，哇，市場權重。給 Solidity 10% 是好事，那才是他們該拿的，對上實際拿到的 7%。我們用 Octant 的實際結果決定交易者損益，但也可以把交易者給的分數當成另一套權重。這塊我不多講，技術細節在我們寫的論文裡。謝謝。希望各位喜歡這場。有人要問嗎？So we ran a prediction market of who's going to get how much money, and actually the Octant organizers and others also said that they prefer the prediction market results. Like when we posted this on Twitter, everyone said, wow, the market weight. Like giving 10% to Solidity is like a good thing, you know? That's like how much they should get compared to the actual 7% that they received. So we use the actual results of Octant to determine profit and loss for the traders, but we can use the traders' score as an alternative to just the market thing. So that's the thing. I won't get into this. This is like very technical in a paper we wrote. But thanks. Hope you enjoyed the talk. Anyone have a question?

17:36我很喜歡這場。你提到倒數第二張、再前一張。對，這張。如果我是像 Teku 這樣的專案，我不是有誘因去把自己的預測市場抬上去，做成自我實現？我在自己的市場上下注，可能還因為多拿到補助而賺更多。對，二次方融資尤其看過這種問題，你可以投給自己的專案。最大的差別是：二次方融資裡，操縱幾乎沒成本。I really liked your talk, and I was thinking when you mentioned the second last slide about the one before. Yeah, this one. Would, if I'm like a project like... Teku, wouldn't it be in my interest to, like, bump my prediction market to have, like, a self-fulfilling prophecy? Like, it might be, I might make more money betting on my own market because I might get more funding. Yeah, and I think we saw those problems happen in quadratic funding especially, where you could, like, vote for your own project. And I think the biggest difference is that in quadratic funding, there's no cost to, like, manipulate.

18:23操縱沒有成本。我開很多帳號，投給自己的專案，就拿到更多錢。被抓到最糟也只是票不算。所以操縱沒成本。但這裡如果 Teku 想把市場抬到 17%，任何交易者都可以反向做：Teku 絕對沒值那麼多。我們真的看過一個市場，一個叫 ACT 的 repo 從 1% 衝到 10%，其他交易者覺得絕對不值 10%，又打回 1%，最後市場結算大約 0.7%。買在高點的人虧很多，這其實是好事。Like, there's no cost to manipulation. I create many accounts, I can vote for my project, and then I get more money. And if I'm caught, the worst that happens is that my vote is not counted. So there's no cost to, like, manipulation. Whereas here, if Teku tries to manipulate the market to also move it up to 17%, then any trader can counter-trade them, saying that, look, Teku is definitely not going to be worth this much. And we actually saw that happen in one of the markets, where a repo called ACT, from 1%, they shot up to 10%, but then all the other traders are like, this is definitely not worth 10%, and they got it back down to 1%, and it finally resolved the market at, like, 0.7%. So any trader who bought it, like, lost massive amounts of money, which is, like, good. Yeah.

19:08提問：市場可能不識 Rotki，評審問責也很弱

19:08下一張，例如這是給質押者的，對吧？質押者可能真的想要 Rotki，因為 Rotki 是好產品，但市場可能根本不知道那是什麼。市場只認識 Solidity。這種時候市場沒有比真正知道自己要什麼的人聰明。他們要 Rotki，市場不認識。所以是雙向的：市場可能影響結果，市場也可能對真正想要的東西很天真。我覺得這也是為什麼要重複跑。下一輪 Octant，市場會學，因為大家在 Rotki 那些上面虧過，就不想再虧。And in the next slide after this, you have, like, for example, this is for stakers, right? And stakers might actually want Rotki, right? Because Rotki is a good product, but the market might not know what the hell it is. And the market just knows Solidity, so in this case, the market isn't as smart as the people who want what they want, right? Like, they want Rotki, and the market doesn't know what it is. So it goes both ways, right? Like, the market could influence the result, but the market could be naive to what the result really wants. I think that's why it's also important to run repeated iterations. So in the next Octant round, the market is gonna learn, right? Because everyone made such a loss on Rotki and all of that, so they're like, yeah, we don't want to make a loss again.

19:59多跑幾次，市場就會學。而且有另一套權重也好。不然你只能接受一套權重，有替代可以對看比較好。模型建造者那邊，很多人在做會實際交易的 AI 模型。他們不一個一個看專案，而是做一個看程式碼庫的 AI 模型。這個例子裡看了 Rotki 的程式碼、Solidity 的程式碼，模型就輸出這個。模型能納入的資訊層級跟人很不一樣。預測市場可以把不同 AI 的輸出合在一起。懂 LLM 的人知道，這叫 ensembling。So then the market ends up learning if you run it multiple times. And even, like, just it's good to have an alternate set of weights. Like, otherwise you're forced to accept one set of weights, but it's good to have an alternative that you can look at. And, like, the other thing about model builders is that a lot of them are building AI models which actually give, which actually trade. So they don't look at each project. They just build an AI model which looks at the codebase. So in this case, Rotki's codebase was looked at, Solidity's codebase, and that's what their model outputs. Because models can consider very different levels of information from a human. So that's, like, a way of using AI to kind of, like, a prediction market can just combine different AI outputs. It's called ensembling, for those who know LLMs and all of that.

20:49所以這只是一種方法。預測市場是把不同 AI 做 ensembling 的方法。還有問題嗎？嗨。想回到上一張。對，這張。我好奇，依我理解，評審是決定賭徒賠付的人。但你好像覺得評審可能錯。評審比較像足球賽的裁判，或是給我們地面真相的 oracle。So this is, like, just a method. Prediction markets are a method of ensembling different AIs. Any other question? Hi. Yeah, I wonder, can we go to the previous slide? Yeah, yeah, this slide. So I'm curious, to my understanding, the jurors are the people who determine the payout to the gamblers. But then I think you mentioned that you seem to suspect that maybe the jurors are wrong. But then I'm curious, like, the jurors are sort of like the umpires in a football game, or they are the oracles who give the ground truth to us.

21:44那誰來確保評審被問責？誰來確保評審被問責？這很難，因為就算在沒有預測市場的一般世界，補助評審的問責也很有限。我是資助者、有員工，員工做了錯的決定，也很難抓住他們說你做錯了。一般世界裡，評審問責本來就低。這個例子裡，評審票的來源是 Vitalik 提名兩個人，他們做比較，再提名兩個人，再提名兩個人。整棵樹就長出來。從資助者那一個人開始。So, yeah, like, so I wonder, like, who makes sure the accountability of jurors? So I wonder, like, who makes sure the accountability of jurors? Right. Yeah, it's a very hard question because it's like even in the regular non-prediction market world, grant evaluators have very limited accountability, right? Like, if I'm a funder and I have staff, if my staff make a wrong decision, it's very hard to hold them accountable and say, you made a wrong decision. So even in the regular world, evaluators have very low accountability. In this case, the juror votes, like the way we got the jury was Vitalik nominated two people, and they did comparisons, and they nominated two, and they nominated two. So we grew out our entire tree. So you start with one person who's the funder.

22:30評審分數一平均，市場有時比平均更一致

22:30他們提名兩個人，那兩人再提名兩人，再提名兩人，整棵樹就長出來。評審票就是這樣建的。但這裡我覺得，問題甚至不是評審對或錯。你把評審資料平均，因為不是一個人在評，很多人在評，再把分數平均。一平均，很多東西就破了。很多評審會說我完全不同意這個分數，因為他們只是 50 個評審裡的一個。平均這一層，常常有人不同意。細節很多。平均之後大家不同意平均結果，但每個人只給一個資料點。市場有時比很多評審票的平均更一致，因為市場遵循另一套組合規則。They nominate two people, they nominate two, they nominate two, and you can grow out your entire tree. So that's how we built the juror votes over here. But in this case, I think, like, the jurors, it's not even a question of them being right or wrong. It's like you average the juror data because it's not like one person is evaluating. There are many, many people evaluating, and you average the scores. So once you average it, a lot of things break. A lot of jurors are like, I don't agree with this score at all, because they were just one juror out of 50 jurors. So there's that also, the level of averaging, which, like, which people don't agree with often. So there's a bunch of nuances here. Like when you average out, people don't agree with the average results, but they're all just giving one data point. So the market can be sometimes more consistent than the average of many jurors' votes because the market follows certain, like, different combined, like, combination laws.

23:27時間到了，抱歉。抱歉，時間到了，但各位之後可以再聊。好，謝謝 Devansh 這場很好的演講。給他掌聲。Time's up, sorry. Sorry, time's up, but you guys can talk, yeah, have a discussion later. Okay, so thanks, Devansh, for this wonderful talk. Yeah, so let's give him applause.