Core Level Selection

[Drakona]

Well, on the wrath thing, here's who all has played it . . .

Jediluke vs melvin 20 - 2 in wrath
Morf vs Lady Silver 20 - 11 in Wrath
Drakona vs psion 20 - 2 in Wrath
Jediluke vs spud 20 - 2 in Wrath
Jediluke vs Drakona 20 - 14 in Wrath
Jediluke vs Drakona 20 - 16 in Wrath
Drakona vs LotharBot 20 - 13 in Wrath
Drakona vs Ferno 20 - 1 in wrath
PFunk (Sloth) vs Ferno 20 - 8 in wrath
Jediluke vs Drakona 20 - 16 in Wrath
LotharBot vs PFunk (Sloth) 20 - 12 in Wrath
Jediluke vs Drakona 20 - 14 in Wrath
Entropy vs A Future Pilot 20 - 6 in Wrath
Jediluke vs Cyrus 20 - 16 in Wrath MCap 6
Jediluke vs Cyrus 20 - 7 in Wrath
Jediluke vs Behemoth 20 - 7 in Wrath
Jediluke vs Morf 20 - 6 in Wrath
Jediluke vs Diablo 20 - 8 in Wrath
Diablo vs LotharBot 20 - 16 in Wrath
Jediluke vs Lady Silver 20 - 5 in Wrath
Jediluke vs Lady Silver 20 - 1 in Wrath
Jediluke vs Diablo 20 - 12 in Wrath
Diablo vs Jediluke 20 - 18 in Wrath
Jediluke vs Diablo 20 - 17 in Wrath
Morf vs roncli 20 - 15 in Wrath
Drakona vs Diablo 20 - 15 in Wrath
Diablo vs Drakona 20 - 15 in Wrath
Diablo vs Morf 20 - 18 in Wrath
sdfgeoff vs PPski 20 - 15 in Wrath
Jediluke vs Diablo 20 - 10 in Wrath x2
Jediluke vs Diablo 20 - 14 in Wrath x2
Behemoth vs Diablo 20 - 15 in Wrath
Jediluke vs Diablo 20 - 12 in Wrath x5
Jediluke vs Drakona 20 - 14 in Wrath x2
Jediluke vs Entropy 20 - 2 in Wrath x2
Jediluke vs Sirius 20 - 12 in Wrath
Jediluke vs Drakona 20 - 15 in wrath x2
Jediluke vs Drakona 20 - 10 in Wrath x2
Sirius vs bahamut 20 - 8 in wrath
Jediluke vs bahamut 20 - 11 in Wrath x2
Jediluke vs PFunk (Sloth) 20 - 6 in Wrath
Diablo vs ERNIE 20 - 12 in Wrath
Vainiac vs Swarthy 20 - 10 in Wrath
Jediluke vs Drakona 20 - 16 in Wrath x2
Jediluke vs Drakona 20 - 10 in Wrath x2
Drakona vs Jediluke 20 - 16 in Wrath x2
Jediluke vs Drakona 20 - 9 in Wrath x2

. . . I guess if that looks like a small group of pilots to you, it is what it is. To me it's evidence that the community as a whole can be interested enough in something new to displace the old. So I don't think the problem is that we're all so attached to core that we won't play anything else. I think it's that we're hungry for something new and good to play, and such levels are rare. And the solution isn't to change how core works, it's to make more good levels.

How to count level modifiers is always a little contentious. The script I have to do the counting sometimes breaks it all out, sometimes it combines x2 and x3 and x4, sometimes it combines different things. I run it different ways depending on what we want to know, so I guess the short answer to why it's broken out right now is that's how I left it last.

We try to do what makes sense, and it's kind of case by case. There was a HUGE kerfuffle in the first season about how we were counting the Athenas, as Athena NH, Athena GV, Athena LH, and regular Athena were all popular, and there was a question as to which one should make core, as NH, GV, and LH together were more popular than regular, but regular was the most popular by far if you ran them all separately. I think we eventually went with LH in the core that season as a compromise, but there sure was a long discussion as to why. Water under the bridge now, as regular dominates.

The season that we had to discuss whether logic x2 should be allowed in resulted from some similar arithmetic; x2 x3 x4 separately didn't make the cut, but if you counted them all as a vote for "X", they did. We asked the community whether that was okay, or whether we should throw it out. They said to throw it out, and it was a good call.

As to why you might count them separately at all . . . I'd say with some levels, the X's make a huge difference. Ascend and Ascend X4 are completely different, I think, and both good in their own way. With some levels, the difference is small. Athena and Athena x2 are practically the same level, and then only way in which Athena x4 is different is that you can't see anything. And Wrath and Wrath x2 are barely distinguishable beyond x2 being slightly better IMO . . . so I would be inclined to count them together.

That's a long winded way of saying, if something's close to popular, we look at all the variants and do what makes sense. I usually run the script in a "combine everything" mode and a "break out everything" mode. It rarely makes a difference, but if it makes one, I'll note it in the core discussion and we can decide what to do about it.

[Cyrus]

I agree with Roncli. The top eight (8) should be automatically put into the next season and the bottom two (2) are voted on by the community.

Everyone who wants to suggest maps is welcome to, and is limited to three (3) map suggestions.

Everyone gets to vote once for one (1) map.

The top two (2) from that vote get put into the next season.

If they are NOT in the top eight (8) by the end of the season they can't be nominated until the following season.


I think this would make everyone happy and not change things too much - my thoughts.


EDIT:

I also want to add that I don't think it's right that challenges & promotions, where the random button is used, should be used to determine the popularity of the map.

[Jediluke]

I discussed the following with Drakona before this forum thread existed.

I told Drakona that I think that anytime 'the button' was used to select a random core level....then that match should not be counted in the statistics for determining if the level should be core at the end of the season....or should count less.

In otherwords ...if you were 'forced' to play the core level...it shouldn't be reflected as a level 2 pilots chose to play thus proving it is popular.

sheeeit, 4 hrs of sleep...i'm finding it hard to make any sense of what i'm typing

[Drakona]

I don't think you can throw those games out entirely. When I agree to a random level, I am partially saying that I'm okay with all the possibilities. I guess it would more propery be 1/11th of a vote for each core level, rather than a full vote for the specific one, but that should even out, right?

[Drakona]

Hold on a sec. I've been saying for a long time we can't measure the effect, but maybe we can...

[Drakona]

So, I would expect popularity -- along any reasonable metric -- to follow a smooth curve, for matches played (or whatever) vs. level, sorted by most popular.



If core levels are getting a noticable boost in popularity -- large enough to disturb which levels are core -- I would expect to see a disturbance in the curve, and would be especially concerned if that disturbance were big enough that it is larger than the typical gap between levels at that part of the graph.



. . .

Here is the actual data. For this season:







For last season:






I won't go back further for now, since last season is when we introduced the promotion challenges, which changes the importance of core, so we can't reasonably expect to see the same effect two seasons ago.

-------------

What do I get out of this?

So, first of all, the levels that are in core can definitely change. Look what Crimson did! It's not as though the barrier is insurmountable.

But more to the point -- am I seeing the core level boost I was expecting? Maybe a little. In some graphs, there definitely is one, on some maybe, on some definitely not, and on some it's hard to tell apart from the noise if it's there at all.

Total matches played chart, E15 looks like it has one, F15 doesn't. The gap on the E15 chart looks like it might cause about 5 ranks worth of confusion. On the F15, no ranks.

On the unique players chart, I see a noticable gap on the F15 chart, worth about 4 ranks. On the E15 chart, I don't see a gap. 0 ranks.

Play twicers E16 . . . that's tough to say, since some non-core levels passed core. Take2 is an interesting case, though, since it's clearly sticking up further than it should. If I compare it to where the curve would fall, it's off by maybe 2 or 3 ranks. Play twicers F15, I don't see a gap. 0 ranks.

Unique pairs, E16 . . . if I ignore take2 again for being weird, I see a gap worth about 4 ranks. Unique pairs F15 . . . if there is one, it's too small to be worth a rank. 0 Ranks.

What gaps did I see? 5 ranks, 0 ranks, 4 ranks, 0 ranks, 3 ranks, 0 ranks, 4 ranks, 0 ranks.

So . . . that could just be noise. A gap in half the charts, absent in half the charts . . . smells kinda noisy. Certainly all the gaps are no bigger than other noisy jumps in the chart, so it is possible there is no effect at all.

Being generous and taking the entire distance as the size of the effect, it is worth at most 3 - 5 ranks to a level's position on the chart.

Now, on some level I am really not concerned about this effect. I think the core levels should be pretty stable over time, and should not be used as a tool to promote diversity (well, anymore than they already are). So if there is a stabilizing effect on them, I do not think that is even a bad thing. In fact, given that we've had levels drift in and out on the bottom end season to season, I'd rather they were more stable than they are. I do not like seeing Black Rose flip flop season to season. I'd much rather it be in until it's out.

And, historically speaking, we've examined levels within about 3 ranks of the cutoff as 'marginal' anyway, and tried to make a reasonable call. For example, this season we have 11 levels rather than 10. Some seasons we've had 8. That the core level effect is as small as the fuzziness we deal with anyway means this effect should not be large enough to affect anything!

However, if we want to be rigorously fair about what people are playing in the selection of core (and I am not sure we should, but if...), this suggests that we should accept levels down to about 5 ranks below what we normally would for consideration in core, and use a separate method to examine those.

I definitely do not like the idea of using a poll, though. I do not trust people voting on forums a bazillionth as much for measuring what people want to play as . . . what people play. Not everyone who plays will vote. People who haven't played in four seasons will vote. People who have only ever played one match and only know two levels will have as many votes as people who have played hundreds. No. The gold standard in my mind for what people play is what they play. It seems very right to me that if you want to change what the core levels are, the method you have to use is to actually persuade people to play something else.

[Morfod]

What I might recommend is if players would like to see diversity in levels in core, is to really get behind one or two levels, collaborate and encourage play. That is how Crimson broke through after sitting on the outside looking in for so long. Most people were exposed to it because Diablo liked to play it. People known to play a lot of different maps added it to their playlist to suggest when their opponent was stuck on where to fight.

Then LadySilver declared it her home, and a while later, so did Cyrus. Three pilots promoting a level is what it took.
Now something similar is happening with Salute. It's a favorite of one active Gold pilot (Diablo) and the new home of an active Silver pilot (PFunk.)

Look to do the same. I see those who are discontent with the core level offerings promoting 2-5 different maps and willing to play others' non-core suggestions. All this does is create a whole mess of one-offs and no concentrated push.
Talk with the active pilots who are in the same boat, and start rowing together.

Same with blackballing a level you dislike. Cyrus did that in Fuzed. Only played under strong insistence (or goading) or when it came up in random. Completely within the rules, and as one of the more active players, definitely affected it's rating in a notable way. It barely squeaked in.

I used to have a policy of making a core level my home for the season. Perhaps I should amend that for next season...

[roncli]

I don't think this topic is very important for the long term health of the ladder. I still agree with Vaniac that some fresh blood should be brought it, and really disagree with Lothar's contention that we're throwing random levels in and that top pilots shouldn't have to adapt. However, in perspective, this is one of those things that's not going to upset me if it doesn't get implemented.

The only thing I use the random button for is for challenges and for when me and my opponent's ability to decide on a level is like me and my wife's ability to decide on a restaurant. "What do you want?" "I don't know, what do you want?"

With that said, I do like the idea of spotlight levels even better than messing with the core level formula. We talked about something similar in the past, where we put up a list of levels that are suitable for trophy matches. But what about just plain old regular matches? We could have something automated that spits out the top 5 non-core levels being played every so often, and even highlight a new level by a new designer. Drakona, Seawyrm, and Swarthy are all making levels TODAY, it would be great for level designers to have an avenue to get some vision on the DCL other than just being told "promote it by playing it".

Morfod, we REALLY need to do that level review thing you suggested last summer. I think that would be pretty fun.

[roncli]

[roncli]