@Ryannn what is the criterion by which you are going to resolve this question? The leaderboard for BIG-Bench Lite hasn't been updated in a long while, and does not list either model. The BigCodeBench Leaderboard (hard) is regularly updated, it currently lists an un-specified model "Grok-Beta". Does the BigCodeBench leaderboard suffice for market resolution?

@Ryannn any guidance on how you might want to resolve this?

@aleven I will gladly take guidance on this

@Ryannn I can outline what informed my bet: in the family of BIG-Bench scores, I could only find the leaderboard on CodeBench I referred to in my earlier comment. There Grok is behind, hence my bet on NO. I could not find other BIG-Bench leaderboards that includes both models, but maybe I have missed them. For example, this leaderboard has BIG-Bench Hard as a benchmark but does not feature GPT4-Turbo, nor Grok-2.
That said, I would ask other traders / mods for guidance on the resolution, given that a BIG-Bench score as you may have intended at the time of creating this market does not appear to exist for Grok-2.

Cool question!

Well they claim Grok 2 beats gpt 4 turbo on mmlu. We will need to wait for API access to do an evaluation on big bench.

I think the question should specify the GPT-4 version.

@HannesLynchburg agreed

some clarity would be good. I've been assuming gpt4's release version.

@HannesLynchburg GPT-4 Turbo

https://www.ft.com/content/19cce4b5-b2dd-4c1e-a109-b500e504dbb6

Subscribe to read | Financial Times

News, analysis and comment from the Financial Times, the worldʼs leading global business publication

@RyanKidd Paywalled, what does it say?