I think one thing is to be proactive shortly after market creation, ask for any and all clarifications you can think of as a trader, even in markets you aren't going to be a trader in. A stitch in time saves nine

I'm all for my friends quitting addictive online platforms!

But maybe the lesson to learn is that markets on the names of things are hard to properly specify, and you can never be sure that a market will be resolved according to your understanding.

Thanks for all the comments!! Update:

I think the most likely possibility right now is that I will probably leave the platform but won't post a (mostly) negative thread about it, more likely a neutral one. (This would mean the market resolves N/A.)

There's a lot of discussion happening here in the comments. That there is the will & energy for having productive discussions around this seems clearly good to me.

To people asking (me personally) what could be improved:

I think there needs to be a higher bar for creating markets. Or a minimal vetting process. And/or make the vetting of creators easier and more transparent. (I agree that mod intervention brings other problems.)

I see lots of markets on here where the creator didn't spend more than a minute researching the specific topic their market is about. In an obvious way. I think that's just really bad. A lot of (new)people can't easily tell good creators from bad creators! And if I'm not an expert on a topic, I can't tell that a market creator is likely not reliable. (Also, it takes a lot of time to personally vet someone properly!)

Last year, there were two very big markets on “Will Google release Gemini in 2023“. One resolved yes, the other resolved no. Both markets had extremely similar resolution criteria. That's insane!

(One creator just decided that the ultra model was the one that counted, while the other took it more literally. None resolved N/A.)

That's bad & It's especially really bad for newcomers. (Which I'm not.) I think this heavily discourages new people joining the platform. Why would I start participating in a platform where I sometimes need to personally know the people who create markets? And the platform doesn't even have a good way of helping me get to that important info. Or doesn't tell me how important this info is. It's just a built-in disadvantage for new people. And this is not even clear to most new people.

(I think the assumption most people would have is that wrong market resolutions basically don't happen, because there should be a good correction mechanism. And if they happen, they should be a mini-scandal at least. I think that‘s a reasonable assumption. But people might disagree with me here.)

Edit: TLDR

• Make it more obvious to newcomers that this is how the platform works (markets do just resolve falsely sometimes). This isn't obvious to newcomers.

• Make it harder for bad market creators who don't care very much. (This is probably the hardest bit. But I think creating a market is something that should really be taken seriously, if you want people to treat it seriously and bet on it. More hoops to jump through is probably good. Take hoops away only if they have a good track record.)

• Make it much easier to access this information for newcomers. Right now this info is not easy to find. One goes through… past markets? It takes too long to be a realistic option. If somebody creates bad markets, there needs to be some cost. Maybe have a reputational market. (I know this is also tricky & gameable, but making big mistakes with the truth also shouldn't be completely free on a prediction platform.)

@JonathanMannhart without any intention of irony, I really think personally I would resolve NO, but it is of course up to you. EDIT: reread the description, and N/A is clearly rights

@JonathanMannhart

I think there needs to be a higher bar for creating markets

What would you envision as a "bar for creating markets"? (since currently there's no bar, it's not obvious to me how to raise the non-existent bar).

@Ziddletwix you didn't ask me, but I think I have seen markets created by people who haven't done much betting.

I agree that the market linked was resolved incorrectly. I also lost mana on that market and considered chiming in, but I felt like the market owner resolved it in good faith in their own eyes.

• I had another market very similar to that one which traders unanimously agreed on, though I made the wording a bit more malleable because I knew the naming might be an issue
  

• I still think allowing market creators to resolve the way they see fit is Good

In my experience, it is far far more distressing to be harassed over your market resolution than to be burned by a resolution you disagree with. I've multiple times resolved to market consensus rather than what I thought was Correct™️ to avoid drama, and I wish this community were even less inclined to bully market creators over pedantics. One or two bad experiences with traders getting way too butt-hurt over an IMO correct resolution have made me a lot less inclined to create markets.

@probajoelistic Oh my goodness, yes, can confirm that bullying market creators is by far the biggest problem. Just recently we had a case of a creator trying to answer a clarification question and traders attacking him and he was like, ok, I'm out, the moderators can resolve this market.

@dreev bullying creators would not happen as much If markets had sensible and clear criteria. (and if it did you could just laugh in the bully's face and point at your reasonable criteria).

You are presumably talking about this one, which, howerver much we can respect it's creator, is a pretty low quality market. Any market that resolves to the creator's "best judgment" that becomes as big as this one did is a recepie for distributed unhappyness.

Skimming the market you link makes me think it's legit ambiguous, but I don't know Apple's naming scheme well enough to have an informed opinion on what reasonable assumptions would be.

In general I agree with your critique of sloppy/low-effort markets, and I know I myself am guilty of making a number of underdefined markets. I don't think this is Manifold's fault though, it's just a natural consequence of letting anyone create a market.

It looks like the context in which this market was created is that OP lost mana on a market that was resolved technically correctly but in an unsatisfying way (/FranekZak/what-will-apple-launch-on-february).

I think litigating the object-level case is instructive: This was a free-response market on which product would be the main one that Apple was launching. A user added an answer for "iPhone SE" but not for "iPhone" in general or the "iPhone 16e" in particular. Since the iPhone 16e was the main product being announced but there was no answer for this item, the creator chose to resolve to "Other". A few observations: Firstly, this was the only technically correct answer option. Resolving to the iPhone SE is incorrect because that's simply not the name of the product line (in another world, Apple could have called it the "iPhone SE 2"). The fact that the creator clarified that "if it is impossible to clearly determine which announcement is the main one, the outcome may be marked as N/A" is irrelevant because the main item was clearly the iPhone 16e. Secondly, we can understand why users betting on iPhone SE are dissatisfied: the iPhone 16e is spiritually the successor of the iPhone SE and it was the only (named) iPhone option available to bettors. Given the circumstances, these bettors argue for N/A-ing the market. This option seems reasonable as well.

Now we're getting to the crux: OP's problem is not one of market resolver competence or lack of proper appeals process, but of jurisprudence—the question of how legalistic versus pragmatic market resolution should be. This is a hard question. And it's a hard question because because it's a problem with language, reality, and human decision-marking—the fact that questions often contain ambiguities or false presuppositions—and not just the specific mechanisms of this site.

Manifold has traditionally tried to sidestep this question by defaulting to the market creator's standards rather than trying to impose one uniform standard on everyone. This approach has its pluses and minuses.

Interestingly, the solution OP suggests of resorting to more well-defined dispute channels like UMA would in my mind be even more likely to lead to legalistically-correct-but-unsatisfying resolutions like the one he's upset about here. To get to the outcome that he wants, he should be advocating for greater mod discretion + less legalism. I am open to moving more in this direction, but everyone should understand that there are trade-offs with either approach.

@SG Sorry for the lecture...

@SG One other object-level comment: markets on product releases are kind of notorious for contested resolutions. Companies change the name or feature-set of things they've previously announced all the time. Sums-to-one markets should probably be avoided...

@SG more mod discretion and less legalism instead of a UMA would further distinguish Manifold from its competitors.

Mod abuse is much less of a risk on a play money platform than a real money one. With a real money platform, a mod (or mods) abusing their powers can exit with something that has value outside of the platform. With play money, the abuse devalues any potential gains, since the play money's value is ultimately derived from the platform's legitimacy.

More mod discretion is also more in line with manifold as a UGC platform -- those always need strong moderation to be successful.

I think it was a founding principle of Manifold that market creators have full autonomy. This was softened at some point and moderators can and do override egregiously bad resolutions, but I'm wondering if it would all feel differently if you just kind of embraced that founding principle. Namely, if you don't have a specific reason to trust a market creator, just don't bet in their market. If it's a question you're interested in, you could even create your own version of the market. Some people don't like it when creators bet in their own markets -- that's a whole separate debate, I guess -- but I personally think it's fine. I also don't think it's too hard to get a sense of when you can trust a creator. Some you may know outside of Manifold, the founders of Manifold themselves have tons of interesting markets and of course have every incentive to be fair in their resolutions. And you can can check creators' ratings and peruse their other markets to get a sense of how much to trust them.

In short, maybe this all feels different if you treat trust in creators as not presumed but earned? (Personally I do tend to presume trust and haven't been burned by that, but I accept that you have been.)

@dreev that's exactly the problem. Reputation only works for old hands. I've been around since the beginning and I know who NOT to trust (and then I still bet because it's fun) but new users have no idea and get burned and feel like they were scammed. And if you from your mountain of net worth don't understand that's a problem, I don't know what to say

@dreev

founders of Manifold themselves have tons of interesting markets

The founders of manifold are sadly the very creators I avoid. Did you not see the manifold love markets?

Do you have any specific thoughts on how to improve things?

@SG I think the policy of almost absolute deferral to market creators when it comes to resolution is bad. Markets have network effects and "fork it or don't trade" is not really a satisfactory answer. Even if you ignore the network effects and people did just "fork it or don't trade", the outcome would be many near dupe markets, which is still an awful UX.

Having a real dispute resolution mechanism would address this. Either something like UMA or a more empowered mod team.

@SG Also is there some way to view the edit history of a market's description or title? I assume there is but I can't find it. And if there isn't, that should surely be a thing.

@cthor yup 3 dots -> see info -> history. doesn't perfectly handle some complicated market changes but for e.g. changes to the title it's good

@cthor

I think the policy of almost absolute deferral to market creators when it comes to resolution is bad...

Having a real dispute resolution mechanism would address this.

I think people who want Manifold to move towards more aggressive mod interventions in markets should think carefully about the practical details of how that would work? (Because I struggle to picture it.)

Say someone pings me (as a mod) and argues that a resolution is wrong. Currently, it's pretty simple—I only step in if it reaches a quite high bar of "obvious misresolution". What should the threshold be instead?

One option: if I am convinced by their argument, and I agree that a different resolution is better, I should step in and change the resolution.

In practice this seems like a quite strange system? Currently, bettors know that resolution is determined by the creator, and can bet accordingly. Under this system, in the event of ambiguity, it seems what matters is "can you find a mod you can convince?". Mods aren't particularly qualified to evaluate most markets. And as a trader, I do not like the idea that in the case of ambiguity, I can't even sure who will be doing the ruling—instead it'll just be whatever mod bettors can convince their side is right.

That sounds a bit like a strawman but I also don't really see what the alternative is. (and the solution has to account for practical logistics—50-100 markets are created each day, disagreements over resolutions are quite common, this plan needs to account for pretty limited mod bandwidth).

Either something like UMA

What aspects of UMA do you think Manifold could/should adopt? Polymarket is a very successful site, but I do not hear almost anything positive about UMA in practice (Polymarket creates the markets itself, and then issues clarifications that UMA always follows for basically all contested resolutions, I don't see how that can scale to 75 user-created questions per day).

@Ziddletwix the issue mostly stems from something occuring that the creator did not account for, because writing good descriptions is hard.

In cases where they're active, the market creator then acts honestly and resolves to something that seems like a most legally correct interpretation of their own unfortunately written description. But as mentioned above, the most legally correct interpretation is not always the most spiritually correct one. This is a stressful situation to be in, leads to many sore feelings for everyone involved, and probably discourages the creator from making any markets again. Some markets I can recall like this:

https://manifold.markets/mattyb/will-mariahs-all-i-want-for-christm

https://manifold.markets/JRR/vp-debate-will-walz-or-vance-win

In other cases the creator is absent and this causes mass confusion for traders and mods alike.

During the Paris Olympics, markets asking "Will [USA/China] win the most gold medals?" and various variations of "most medals" resolved to all of YES, NO, 50/50, and N/A depending on the whims of the market creator, and many of these resolutions were not clarified until after ties had occured.

The various markets regarding whether there was or will be a US government shutdown are a similar case.

What it would look like in all these cases is mods looking at the markets in question and deciding that the creator is out of their depth and so taking over responsibility for it, giving the platform as a whole a single definition for certain things (e.g. the definition of "most medals"), and clarifying and resolving as appropriate to the spirit of the markets. This gives the platform as a whole a more consistent resolution philosophy (that of the mod team's discretion) and obviates the need for traders to pay close attention to the finer details and reputation and history of every single person whose market is traded on, and it allows them to trade somewhat confidently on newb created markets that aren't written particularly well, because they're confident that the mods are there to fix things if something goes wrong.

It also allows newbie market creators to have mods take a sticky situation off their hands as a rule rather than the exception. (As a parallel, the current situation is sort of like a sports referree asking the opposing team if they'd like the rules to be enforced. When this happens, it puts the player being asked in an unfair bind: to say yes would be unsportsmanlike, but to say no is to in effect say the rules do not exist. The issue lies in the referee abdicating their duty.) Mods are more experienced with market resolutions and better equipped to handle these situations, so them taking on that responsibility should be better for everyone involved.

It could also just be a mod uniltaterally making a clarification and the creator thanking them for the help. (This is not the same as the current process of a mod first asking "Hey I think you should add clarify this? Might I suggest A, B, or C?" because then traders still need to wait for the creator to respond before trading confidently.)

(Alternatively, they get upset, never come back, and badmouth Manifold elsewhere. It did happen a lot with sweeps that a lot of people got upset about their market being "taken over" by sweeps. But I think there is a key difference here. The sweeps markets that were overtaken were generally good markets, and the creators who were upset are rightfully proud of running their markets well independently. Whereas most markets needing to be overtaken are created by either newbs or absentee creators.)

Currently, bettors know that resolution is determined by the creator, and can bet accordingly. Under this system, in the event of ambiguity, it seems what matters is "can you find a mod you can convince?"

It is much easier to get a grasp on the predilictions of a collective than a lone wolf. For example, if I were to ask, "What does the Republican Party think about abortion?" or even "How will Congressman from Redville, Red State vote on this abortion bill?" you could give a confident answer, whereas with "What does this random Republican voter I pulled off the street think about abortion?" you would be far less confident.

Procedurally this look like a mod looking at whatever queue exists, making a decision, running it by the other mods, and then if none of them disagree with it, taking action. If there is some disagreement, it's resolved internally. If mod shopping were an effective strategy here, that suggests the mod team not only have an incoherent resolution philosophy but that they're not voicing internal disagreements and just rubber stamping everything.

this plan needs to account for pretty limited mod bandwidth

The current timidness of mods generally results in unnecessarily delayed clarifications to wait for an absentee creator which only leads to further issues. For example, in https://manifold.markets/GeorgeVii/will-this-yudkowsky-tweet-hold-up-2, I'm not privy to why the mods decided on N/A, but it looks a whole lot like the market should be YES (creator had already clarified that it was NOT required by EOY 2024 that videogen must be that good enough yet, merely that it appears to be on a trajectory roughly implied by the tweet, i.e. that videogen being good enough in EOY 2026 sounds plausible at EOY 2024) and it's only the consequences of this policy that lead to the whole kerfuffle and an eventual N/A, because by the time mods took any action it was too late and there was no good options.

I suspect that mod workload would actually decrease if mods took over more markets, because resolving unambiguous markets is a lot easier. The longer things are left uncertain and the more back and forths with the creator are needed, the more work there is to do.

@cthor For now I'll just focus on the narrow points about mod workload/practice (because it may be somewhat opaque to those outside & I can attest to personal experience).

I suspect that mod workload would actually decrease if mods took over more markets, because resolving unambiguous markets is a lot easier.

This would not be true (nor do I think it would be close). There are 50+ markets created each day. As an exercise, I would look through the list of new markets, sample a few at random, and ask yourself—"does this have sufficient resolution criteria to avoid ambiguous edge cases?". A substantial percentage quite obviously do not meet that bar.

Manifold creators are bad at writing descriptions. Manifold traders don't seem to particularly care if descriptions are good when they place their bets (make enough markets and you see how insensitive they are to that sort of quality). The result of these two factors is that the amount of work required to proactively address all major ambiguous cases on these markets would be gargantuan. You could 10x the mod staff & you wouldn't make a dent (in particular because there wouldn't be much appetite for that obviously thankless work).

I would use the market linked to by the OP as an instructive example. It isn't at all surprising that it had a contested resolution—this happens all the time with product names/announcements, anyone glancing at the market could guess that there would be issues. But what exactly would the fix have been? It is genuinely hard to write a good version of this question. There was no easy, preemptive fix that could be applied, if only had a mod been empowered to step in.

When I browse the new queue, I see an endless list of "potential future ambiguous resolutions", and there's no trivially easy fix for most of them.

On a lighter note, I will just say:

It could also just be a mod uniltaterally making a clarification and the creator thanking them for the help.

Of all the fantastical proposals people are outlining in these comments, this might be the most fantastical. I assure you, this is not generally how it goes down!

@Ziddletwix if mods were only actioning stuff on request (probably by a trader), how many of those daily would you expect? It seems like in the vast majority of cases things go fine despite bad descriptions and no intervention is needed nor asked for, since as you say the traders also largely don't care.

I assure you, this is not generally how it goes down!

ehehe, that one was a bit tounge in cheek 😂

@cthor

if mods were only actioning stuff on request (probably by a trader), how many of those daily would you expect? It seems like in the vast majority of cases things go fine despite bad descriptions and no intervention is needed nor asked for, since as you say the traders also largely don't care.

Yup that makes more sense—I interpreted "proactive" to be more broad (i.e. not just when a trader flags it), because that's the only way to actually avoid these regular issues (and that would be an impossible workload). If it instead refers to just "when a trader flags an ambiguous point, mods step in rather than waiting for creators", then that is comparatively manageable (FWIW well beyond what mods could currently handle, but like, not impossible).

While more feasible, I'm not convinced that's a great system. First, it doesn't address most actual reasons that people complain. Again the inciting incident here (iPhone SE) is illustrative—this wasn't a case where there were clarifying questions asked before the announcement that could have avoided this. There are some cases where if mods were empowered to address trader questions and issue clarifications, future disputes could be avoided. But those are nonetheless a distinct minority of disputes.

Second, I personally just place a reasonable amount of value on creator autonomy. When I write a question, I would prefer to have the opportunity to address clarifications myself, because I know what my intent was and in practice I have probably thought about the question more deeply than whatever mod glances at it from the mod queue. In practice, many creators are inactive and/or have given zero thought to it, so the median mod clarification is probably higher quality than the typical outcome when you wait for the creator to handle it. But I think something meaningful is lost when that becomes the default, because the best creators can run their markets much better/more thoughtfully than however the workload gets distributed in the mod queue.

(I also think "workload constraints" are real here—I am personally willing to help out in the "janitorial clean-up" duty part of being a mod to improve the site a tiny bit, but I have no real appetite for the thankless work of answering thorny clarifications that traders raise after creators write bad questions, and then having the creator feel annoyed that I took over their market and misunderstood what they were going for—autonomy matters, personal ownership over my markets is part of what makes this site fun and part of why I have spent so much time running markets. When it comes to obvious misresolutions, the costs/payoffs are wildly lopsided, so it's an easy win. When it comes to proactive adjudicating of edge cases flagged by traders, you get a laborious solution that won't handle most future disputes while also undermining some of what makes Manifold special to me).