Ravnica Allegiance First Look: Ravnica Remixed

February 1, 2019 By Dan Troha 4 Comments

I think it’s safe to say that everyone had pretty similar expectations about Ravnica Allegiance going into its release — it would look very similar to Guilds of Ravnica for limited.

After all, there are five guilds, each with one mechanic. There is another cycle of CCDD cards like Nightveil Predator. A guaranteed guildgate in each pack, 24 in the whole draft. Powerful gold commons.

So does it pan out the same way? Or can the same framework lead to a different draft environment based on different mechanics or cards?

We analyzed a sample of 88,000 simulated Draftsim drafts from last week to find out. This gave our users enough time to play with the cards at the prerelease a bit and have more knowledge about interactions. For the analysis, I’ll be using most of the same thought process and visualization methods from our last article. Be sure to read that one too so you can understand both sides of the story.

As for my own take on the format, I ended up practicing quite a bit in preparation for Grand Prix/MagicFest New Jersey. And I learned a heckuva lot more about the format. That said, there wasn’t a ton of variety in what I drafted:

2-1 the draft at #GPNJ. 9-3 now. #theforgottenGP pic.twitter.com/EOtzUOv8Hg
— Dan Troha (@dantroha) January 27, 2019

Orzhov aggro for another 2-1 and an 11-4 finish at #GPNJ #theforgottenGP pic.twitter.com/52L4Yu6MYY
— Dan Troha (@dantroha) January 27, 2019

Personal experience aside, let’s see what you all in the hive mind came up with. What’s the data have to say?

Synergy

We’ll start with Draftsim’s signature visualization, the synergy plot. Remember that this graphic represents how often a particular card is drafted alongside another card. We introduced this visualization originally in this article.

The cards in each color cluster together because you always draft them along with cards of their own color. Cards that go together nicely will have a position that is closer to each other than cards that don’t.

If we start by looking at just the shapes and colors of the visualization, you’ll see that Ravnica Allegiance looks just like the Guilds of Ravnica picture. Bascially only the adjacent colors are different!

The two are fraternal twins…

Here’s a larger version of the RNA image above so you can see the definition a little better:

And that similarity makes sense. Both sets have gold cards that are only in their guild. They both have guildgates that are only in the supported guilds. The card synergies and cross color mechanics all support certain two color combinations. So everything revolves around that color dynamic.

If your reaction is still “So what, they look basically the same?”, allow me to refresh your memory about just how different M19 looked…

The guild-based nature of the new set creates a very strong correlation between which colors are drafted together. In a more traditional set, you don’t see that kind of perfect pinwheel or “flower” shape that we see for GRN and RNA.

For all the gory details of which cards are next to which, you can click on the image below:

Let’s take a look at the color black to get a better understanding of how synergy affects your color preference when drafting a particular card.

I decided to zoom in on the black cards that are on the left side of the visualization above.

We can see a few things here:

The generically good black cards, eg Grotesque Demise, tend to be drafted with one another. Makes sense – if you have good black cards, you want more black cards so you can build a deck.
The generically bad cards like Font of Agonies pull towards the middle because it doesn’t matter what colors you’re in — you take it at the end of the pack. Picking it is almost random.
Rakdos Trumpeter (on the top) is a special case because while it is a black card, it has a red activated ability. You are much more likely to play it in a black red deck, so hence it ends up halfway in between the two colors. Blade Juggler and Drill Bit are also strongly pulling in the red direction because red helps with enabling spectacle, which makes these cards much stronger.
Then on the bottom of the cluster is the cards that are “more white.” This includes either things that make small creatures like Debtors’ Transport or things that pay you off for small creatures like Vindictive Vampire and Spire Mangler.

Other notable outliers

Of course, color overlap and synergy can take a little bit of knowledge about the set to understand too. Blue and white have a heavy concentration of fliers. You could kind of call this Azorius’s “theme” even moreso than Addendum.

And when you look at Windstorm Drake, a payoff for having flying creatures, it has sorted itself all the way to the bottom of the blue pile toward white.

Similarly, Gates Ablaze looks more like a green-red card. I think there are two reasons for this. First of all, Gruul has more big, dumb creatures that are unlikely to die if you cast a small Gates Ablaze for two. Secondly, green is often the basis for the “Gates Deck” that relies heavily on the common Open the Gates for mana fixing. So it stands to reason that a card that incentivizes you to have gates would be clustered near green.

Rumbling Ruin is also halfway to red-green. Gruul’s mechanic, Riot, often places +1/+1 counters on creatures and this guy rewards you for doing just that. The “can’t block” ability is even more relevant when you have a bunch of large creatures that are suddenly able to get by a bunch of small chump blockers or Gruul’s biggest enemy — deathtouchers.

Color Pairs

Next up, let’s take a look at the relative popularity of each color drafted. The darker a color pair is, the more popular it is:

A reminder here, the x-axis represents the primary deck color — the color with the most cards drafted at the end of the draft. The y-axis represents the color with the second-highest number of cards in the drafter’s final pool.

The first thing you might notice is that just like in GRN, the off-guild color pairs aren’t drafted at all! There are just too many powerful on-color guild cards to ignore the guilds altogether.

So our most popular guilds ended up being Rakdos (22%), Azorius (22%) and Orzhov (21%) with Simic (17%) and Gruul bringing up the rear at 14%. This certainly resonates with my experience online and in person.

If I had to guess, I’d wager that the strength of black is related to the strength of its commons. Grotesque Demise, Blade Juggler, Dead Revels, and Ill-Gotten Inheritance are all pretty exciting Magic cards at common. And the next best cards after them are quite solid as well. The relative depth and power level of the black commons could be one explanation for why its guilds are so popular.

Unfortunately, this analysis does not take into account three color decks. From what I’ve seen so far, many many decks in RNA are composed of two colors with a splash or some that are straight 3-5 colors. It might be a good opportunity to try and capture fully completed decks in the future so we have more information about a deck’s final configuration.

The guild color wheel – a new visualization

On the topic of three-plus color builds, we were curious if some guilds or colors are more “dense” and encourage players to commit to a narrow strategy, compared to other guilds or colors that would encourage reaching out to neighboring guilds.

What if, say, Gruul worked better when combined with another guild, but Simic worked better as a narrow Simic-only build?

To quantify this stuff, Arseny created a “Guild Color Wheel,” ordered with the colors in adjacent guild order (W/U/G/R/B) for RNA. He encoded these colors as angles, with muliticolored (guild) cards assigned angles between the “pure” angles.

For example, you could code white as 0, blue as 1, and Azorius cards as 0.5. Then you calculate the average “angle” of each pool/deck, and the “angular breadth” of each deck. Most decks lie directly on the “guild petal” for their respective guild because they are largely made up of cards from the guild’s two colors.

The longer the petal is, the more popular that color pair.

While I think this looks really cool, it doesn’t look very edgy or revelatory. Still, you can observe how relatively unpopular Gruul is because its peak is shorter than the other guilds. This is just a different way to represent the data in the previous tile visualization.

You can also see that some petals are slightly skewed from the “average guild” because one of the two colors provides more cards to support an archetype. Simic is slightly skewed towards blue. Azorius is skewed towards white. But nothing too dramatic.

We can actually estimate the popularity of three-color decks with this plot as well! Picture a W/U/G deck with an equal number of white and green cards. Both those colors would pull the graph equally away from the color in the middle of them — blue. You can see the popularity of the three color decks then by looking at the notches in the center of each of the five colors.

Orzhov/Rakdos (“B) is deep and narrow. The deepest point lies 1.5 steps from the center of the plot. On the other hand, Azorius/Simic (“U”) is wide and shallow, and the deepest point is 2.7 steps from the center. This suggests that Azorius/Simic is 80% more popular than Rakdos/Orzhov.

This seems like it could make sense because Rakdos skews more aggressive (less time to splash stuff), whereas Simic has cards that both can ramp and fix mana.

Top Cards

Next up, let’s take a look at the top cards in the set. What did you all love the most? And who is the villain? The worst of the worst?

This metric was calculated by simply noting the pick number for which each card was selected and then averaging it over the number of times it was picked.

Rares and mythics

Card	Average Pick Number
Skarrgan Hellkite	1
Spawn of Mayhem	1
Angel of Grace	1
Thrash	1.000887
Mesmerizing Benthid	1.001304
End-Raze Forerunners	1.001604
Biogenic Ooze	1.00222
Warrant	1.00394
Incubation Druid	1.004773
Electrodominance	1.005469

These are all very powerful rares, and I think most of them deserve their place in the upper echelon. In fact, many of them are right at the top of my own tier list for Draftsim.

The most suspect inclusion might be End-Raze Forerunners, which I think is a very good and powerful card, but I certainly overrated how easy it would be to cast. You can’t just put an 8-drop into any green deck. Particularly when Gruul is asking you to be playing threats much earlier and there is a good tempo-oriented build of Simic.

Commons and uncommons

Card	Average Pick Number
Trollbred Guardian	1.488131
Flames of the Raze-Boar	1.503823
Grotesque Demise	1.62925
Mortify	1.636954
Skewer the Critics	1.802282
Incubation	1.987985
Summary Judgment	2.196741
Essence Capture	2.436544
Ministrant of Obligation	2.653157
Depose	2.726859

Mostly what we see here is a couple of very efficient creatures and some of the better removal in the set.

Incubation // Incongruity is particularly surprising given its power level — good but not insane. I understand the flexibility of the split cards and know that hybrid mana makes them higher picks. However, I definitely would hope to see Consecrate // Consume and Collision // Colossus on a list of top picks.

The 10 “worst” cards

Card	Average Pick Number
Clear the Mind	11.9236
Rakdos Locket	11.88896
Rubble Reading	11.87844
Gruul Locket	11.75127
Orzhov Locket	11.71442
Azorius Locket	11.61988
Deface	11.61142
Root Snare	11.55006
Simic Locket	11.53424
Expose to Daylight	11.30305

It’s funny how the context of these cards in the format has changed already. We seem to have more people than ever drafting each set right out of the gate. Between Arena, Magic Online, and paper, every nook and cranny and niche strategy in a format is being explored almost as soon as players can get their hands on the cards.

Clear the Mind is an excellent example. At first, it looks completely unplayable — it basically does nothing! Especially in a set without very many graveyard synergies or payoffs, it looks like a card that has no reason to exist. I wouldn’t blame anyone for ignoring it early on in a draft format.

But what if you have a control deck with a million removal spells and no win conditions? Or what if you’re going all in on Dovin’s Acuity? On a recent episode of the Lords of Limited podcast, the hosts outlined a deck where you really need to have one or two copies of Clear the Mind. It’s definitely not a card you should ever be prioritizing, but it does play a critical role in a deck that will sometimes come up.

Also, lockets are much better in Ravnica Allegiance than they were in Guilds. Many of the people I talked to at the GP were happy to run one or even two in a draft deck. I kept hearing the format is “grindy” and there “isn’t much aggro.”

Finally, I think Expose to Daylight is criminally underrated here. There are so many ubiquitous powerful, game-changing enchantments in RNA that you have to answer. Angelic Exaltation, High Alert, Ill-Gotten Inheritance, and the biggest baddy of all, Ethereal Absolution, just to name a few. And that’s not not even considering incidental Lawmage’s Bindings, Sky Tethers, and Slimebinds out of Azorius.

So the lesson here is that people’s expectations are heavily influenced by previous sets. I would expect to see many of these pick averages go higher as the format matures.

Controversial Cards

Look at that mythic that’s way out to the upper right… it’s Kaya

In short, “controversy” means “variance in pick order.” The more disagreement people have about where in the pack a card should be picked, the more controversial it is.

For a more in-depth look at the controversy calculation method, check out the previous analysis article.

Most controversial cards overall

Last time, we had a lot of “weird rares” take the prize for most controversial card. What about for RNA?

1. Kaya, Orzhov Usurper
2. Emergency Powers
3. Cindervines
4 Font of Agonies
5. Smothering Tithe
6. Rampage of the Clans
7. Mirror March
8. Awaken the Erstwhile
9. Lavinia, Azorius Renegade
10. Wall of Lost Thoughts

Yep, that certainly appears to be the case again. And most of these cards are actively “bad,” not just weird. But they have a high complexity level and strange applications, which makes them perfect for confusing people. It stands to reason that our drafters would evaluate the cards very differently.

So why is Kaya such a HUGE outlier?

I think planeswalkers are generally insane bombs. In fact, many of them have been nigh-unbeatable when resolved because usually the only way to interact with them in limited is by having a board presence and attacking them.

But Kaya doesn’t do much at all. There are very few relevant 1 mana permanents to exile. And exiling creatures from graveyards is barely relevant. And what’s the payoff? What kind of ultimate is she threatening?

So I think some people took the card highly assuming “you have to take planeswalkers when you open them” or exercising a little “simulated raredrafting.” But others realized she doesn’t have many applications in a game of limited.

Maybe the card is going to be great in Modern or Legacy, but it’s definitely not in draft.

Most controversial commons

Looking at commons can help us understand synergy and set context dependent cards that people value differently.

If you can’t live the dream IRL, at least you can try on Draftsim…

1. Persistent Petitioners
2. Spear Spewer
3. Rakdos Trumpeter
4. Simic Locket
5. Gravel-Hide Goblin
6. Goblin Gathering
7. Thirsting Shade
8. Azorius Locket
9. Senate Courier
10. Saruli Caretaker

It’s interesting that we have three of the off color activation creatures on here — Rakdos Trumpeter, Gravel-Hide Goblin, and Senate Courier. This is another class of card that looked a little underwhelming at first but it turned out that each has just enough of what you really want to do in its respective guild (trigger spectacle, two drop, blocker with high toughness) to be highly sought after.

I think the most fun example here is Goblin Gathering. I would say that “high variance in pick number” is probably the optimal way to draft this card.

Say you pick up your first copy – you probably shouldn’t prioritize it because one copy is pretty low power. You take it 10th pick. Then you pick up your next one a little higher. And by the time you get to 4-5 copies you might want to first or second pick the card because each additional copy is just so much better.

So depending on what stage you’re in during the draft and how many copies you have, the pick priority fluctuates wildly. You could also say that for the #1 common Persistent Petitioners. However, copies 1-4 are all still pretty underwhelming…

I would be remiss not to mention my own pick for most controversial common — or at least the most talked about — Ill-Gotten Inheritance. Good ol’ IGI finished at #11 on our controversy rankings. This was a very difficult card to evaluate and while some initial opinions on the card were that it was “unplayable,” things have swung in the other direction and it may be well on the road to becoming overrated.

For a very interesting analysis on just how good this card is, check out Ari Lax’s article from his blog.

Wrapping Up

So we still have a lot to learn. After all, this format has only been out a couple weeks and there’s much left to explore. Most of the guilds have multiple sub-archetypes, plus there are all these three color and gates-matter decks.

But I would recommend a few things based on the data above: First off, start by drafting a bomb mono colored or hybrid card (duh). Flexibility is key in draft, and there’s a reason those cards were unanimous, slam dunk first picks.

I really like the most popular guilds of Orzhov, Azorius, and Rakdos. But all the guilds are playable and people are winning drafts with all five of them. So start your draft by taking powerful cards that can go in multiple guilds or be splashed, then move into the open guild (or gate deck…).

While I had a preference to draft Orzhov at the Grand Prix last weekend, that preference didn’t outweigh the necessity to stay open. You’ll win in this set by being set up to play as many powerful gold cards as you can get.

Finally, take a good look at the most controversial cards and make sure you understand why they are either good or bad. They’re likely to be the most confusing to evaluate if you have to decide how good the card is the moment you open a booster. This list is a great place to start if you want to chat with your friends or people at your store about their experiences with weird cards.

And that’ll do it for today! Good luck in any drafts you’ve got coming up — may you build some incredibly spicy and awesome limited brews.

If you enjoyed this article, be sure to sign up for our mailing list to be notified when we publish new articles like this one or release new sets on the site to draft!

New Year, New Features: Major Draftsim Updates

January 7, 2019 By Dan Troha Leave a Comment

Happy 2019 to all of you out there! This past few weeks I hope you got to play Magic with some friends and to avoid eating too much (hint: I did not). While I had a wonderful holiday break, I’m happy to announce I’ve also been hard at work on some new features for Draftsim. I think you’ll be pretty excited about the new hotness added to the ‘ol draft simulator.

It’s been a long time since any major features were added to Draftsim, so the downtime between sets seemed like a great time to make some big improvements. I think these will all make Draftsim even more fun to use and useful.

So follow along as I take you on a tour through sharing your drafts, the new visual card ratings list, foils in packs, and more!

Sharing Drafts

How many times have you finished a draft and wanted to brag about the amazing masterpiece that you just put together? Or needed to get consolation for the life decisions that led to trainwrecking your draft? I see people sharing decklists from Draftsim on Reddit all the time. Unfortunately, until now you couldn’t share your pick-by-pick choices.

We’ve been able to do this using Flooey for Magic Online booster drafts for quite some time. I’ve wanted Draftsim to have a Flooey-like function for a while, but now that time is finally here!

So how do share a draft? First, fire up a draft on Draftsim and proceed as usual. When you finish, a new button will appear:

Share draft button showing after the draft — This new button will now show after you finish every draft. Click it and you’ll get…

Draft log page — A permanent link showing each of your picks that you can copy and share with others or save for later

The button opens a new page showing a complete log of your whole draft with your selections highlighted! There are also a one-click copy button and social network buttons to make the draft as easy as possible to share.

If you want an example of a completed draft, I’ve got one for you to check out here.

So go ahead, give it a try, especially when Ravnica Allegiance comes out on Draftsim later this week. There’s no better way to learn a new format than to discuss the cards with friends or other draft-obsessed folks. And if you need somebody to chat with, Draftsim has a new Discord server that you should check out.

Foils

Example of Foil card on Draftsim — Ooh shiny. I think I’ll take that…

Yes, Draftsim now has foils! In all their shiny, holographic glory.

Masters sets on Draftsim have always had “foils” — the fifteenth card of the pack was generated separately and could be any rarity. But there never was any effect added to the card.

And regular, non-masters sets like Guilds of Ravnica never had foils. This created some issues with rarity distributions because foils change the pack contents quite a bit. Sometimes there are packs with two rares and you gotta make a tough choice. Now we can simulate that!

And if you’re one of those notorious draft-simulator-moneydrafters, you can draft away and take all the pretty cards your heart desires.

Foils replace the last common in the pack and appear in about 1/4th of packs. Actually, if you happen to have any further information on their frequency, it’s hard to find precise numbers on this. So if you know the specifics, please reach out to me.

Ultimate Masters pack showing the place of the foil in the last slot — Masters sets now explicitly show the last card all foiled out instead of a Plain Jane version

Tier List

Draftsim has all this data and all these card ratings, why not do something with it? The ratings have always been quite popular, often with dozens of people at any given time viewing my ratings spreadsheet, especially after a new set comes out.

Now we have something much better – a visual pick order/ratings list!

Pick order list example for Guilds of Ravnica — Introducing the new Pick Order/Tier List page

You can see the list pictured above here.

I decided to go with something a bit more visual than a spreadsheet here, and I think it’s way easier to parse. It’s still searchable like the spreadsheet, so you can quickly find where Draftsim has any particular card rated.

And rather than just a rating number, the cards are organized into classes (tiers). This helps you think about a format more holistically. You can start to see how cards cluster together and approximately where in a pack you can expect to pick them up.

The tiers are still based on the ratings that are used in the Draftsim engine, but I think it’s a lot easier to understand the overall feel of the format by being able to see the relative card positions instead.

Example of the bottom of the GRN list — The dregs…

You can use these pick order lists as a reference when you’re doing a draft on Magic Online or MTG Arena. As I mention in the intro of the list pages, rankings are not a be-all end-all and usually go out the window as soon as you make a couple picks. But if you’re clueless about how good a card is in a particular format, this is a great place to start.

I think lists like this are a great source of discussion and debate. Feel free to share the list and let me know if you disagree with anything! The Discord is also a great place to talk about it too.

My goal is to keep updating this list and Draftsim’s ratings throughout a format in order for you to have a one stop shop to get an overall view of all the card values at once.

Other Small Improvements

Viewing Bot Positions

I’ve had a lot of people ask me “where are the bots sitting during the draft?” and “is Bot1 on my left or my right?”, so I’ve decided to clear that up and make it a little more obvious.

Now, all you have to is click the View Bots button…

Click the "view bots" button to see the bot picks after the draft — Click “View Bots” after you complete the draft to see all the bot picks

… and you’ll see a label with the bot position.

bot description shown under bot title — Now you can see where the bot was sitting during the draft under its name

If you haven’t done this before, I highly recommend checking out what the bots drafted after you complete your draft. It can help you understand why you were passed the cards you were, and also how your picks affected the players downstream from you.

Or it can help explain why you got cut off from red in pack 2!

Pick Labels

Finally, a small addition for clarity is a Pack/Pick number above the pack when you’re in the middle of drafting.

Pack and pick label during the draft — Note the new label at the top of the pack

I find that this helps you keep better track of how many cards are left in the draft and can help you prioritize your picks better. It also works hand-in-hand with the new sharing function because you can note down the pick number and then jump back to that pick after the draft to review your decision-making.

Hope you like it.

Next Up…

Be sure to let me know what you think of the new features, and especially if you see any bugs or issues, I want to hear about them!

This year I’ve got a lot of big changes planned for Draftsim, not only on the draft simulator but also for this blog. So let me know if there’s anything you’d like me to cover more.

But the biggest thing coming out of Draftsim is going to be some new drafting software that I think you’re going to love. If you’re interesting in hearing about it as soon as I’m ready to announce it, be sure to sign up for Draftsim’s mailing list below.

Cheers and I hope you enjoy the new features!

Guilds of Ravnica First Look: Controversy Abound

October 29, 2018 By Dan Troha 9 Comments

By Arseny Khakhalin, Dan Troha, and Bobby Mills

Believe it or not, we’re already four weeks deep into the Guilds of Ravnica draft environment! We’ve had a couple of Grand Prix tournaments, tons of drafting on Arena and Magic Online, and of course there’s still a Pro Tour coming up in a couple weeks.

Much has been made around the internet about ranking the various guilds and colors in GRN. We’ve seen plenty of people say that “green is bad” and have seen others respond with “you just don’t know how to draft green.” So it’s fair to say there’s been a bit of controversy about how to approach the format.

And that’s what we’re going to get into today. Not only will you get some great new visualizations of the overall Guilds draft format, we’re going to dive deep into a new draft statistic we’ve developed: the controversy of a card among drafters.

For today’s article, we took our data from around 118,000 drafts on Draftsim – collected throughout the first week of October. As before, we anonymously saved the cards and the order in which you picked them given the other cards presented to you in the pack.

Synergies

Let’s start with our now-familiar visualization of card synergies. In case you missed our description of how we previously created them, check out this summary. In short, each card on this plot is represented by a point matching its color. The distances between the points reflect how frequently any two cards are drafted together; the more often they end up in the same deck, the closer they are shown on the plot.

The axes here aren’t labeled because the values they represent are not really interpretable. For this type of analysis, only the relative position of different points matters. Colorless cards and nonbasic lands are shown in gray, and multicolored cards are all shown in purple.

First up – drumroll please… the synergy plot for GRN:

Note how each color is closest to its two guild companion colors

Isn’t it lovely? For you to appreciate just how different GRN is from a “normal set,” recall the synergy footprints for all four sets we’ve analyzed so far:

While DOM and M19 were more or less “normal” sets, and RIX/XLN was clearly a tribal block, GRN is built around guilds. What’s the difference between a tribal block and a guild one? What makes the synergy landscape so different?

Two things probably: one is that in GRN an awesome bomb card does not lock you into a guild that firmly. As long as your mana base supports the awesome card, you can still experiment. An Aurelia, Exemplar of Justice you drafted will still mentor cats, if necessary. It doesn’t force you to only draft angels, or mentor cards, as was the case with Ixalan. The second important difference is that two of the tribes in XLN spanned three different colors, while the guilds are all nice and evenly spread, like a flower.

If I told you GRN didn’t look like this, I’d be lion…

With its symmetrical shape and tight card clusters, almost no single card breaks formation. Rare exceptions are the two somewhat lonely red points at the very bottom that are biased towards Izzet. These are, of course, Electrostatic Field and Erratic Cyclops — cards that strongly benefit from being in a spells-based blue/red deck.

And you can see a black card on the very right that is attracted to Dimir: it’s Whispering Snitch, with its surveil synergy.

It really is astounding to look at the way single-colored cards sort themselves out toward their stronger color pairing. In blue, Leapfrog and Maximize Altitude pull towards Izzet while Thoughtbound Phantasm and Enhanced Surveillance push up against the Dimir pile.

You can see a large 2000×1500 px file with all cards labeled here:

A chart showing all the cards in GRN with relative synergies — Click on the image to expand in a new window

And we’ve also got an interactive visualization you can see here:

grn interactive visualization screenshot — Click here to use the interactive visualization

Color Pairs

We also looked at the color pairs that were drafted most often, as we did for M19 in our last post. Remember that for each final pool of cards drafted, we determined the color that was drafted the most (“Color 1”), and at the next most abundant color (“Color 2”). Then we built a heatmap, showing which combinations were drafted with the highest frequency and which were the least favored.

The plot for GRN looks pretty strange, but the sparseness is by design. Nobody is trying to draft Simic this time around, so only some color pairs in the heatmap table are even shaded:

What do we see here? Dimir (and more specifically, Dimir heavily biased towards blue) is by far the most popular. Note how strong the asymmetry is: 20% of decks feature lots of blue spells and some black, but only 7%, almost 3 times fewer, feature lots of black and some blue. Perhaps this is because both blue guilds are considered to be “good,” while one of the black guilds (Golgari) has been widely criticized for being underpowered. So people hedge by starting with being blue-based.

The next most popular guild is Boros (red-skewed), then Izzet (blue-skewed), and Selesnya (green-skewed). Golgari, on the other hand, though it may be strong in constructed, came in last place in our popularity contest.

Interestingly, it took our drafters some time to come to terms with the fact that green is a tough color to draft in this environment. If we look at how the popularity of different colors changed during the first several thousand drafts, we see that green was becoming ever slightly less popular, while black grew on people. The effect is weak, but it is definitely there.

Controversial Cards

We also had a new idea this time: to look for cards that are “controversial” in terms of their relative rank and rating. The intuition is simple: obviously, there are cards which everyone agrees are great and are always drafted first. If you open Dream Eater, you obviously take it, it’s a no-brainer, and it will probably strongly influence you to draft blue for the rest of the draft.

On the flipside, there are cards that are obviously “bad” that no one wants to take. These are the chaff, the very last cards in a booster. There are also cards in the middle that are sort of OK, and which everybody agrees are just OK. Think Devkarin Dissident, for example.

But we thought that there could be other cards that are drafted early by some people and hated by others, thus dividing the drafting community. And wouldn’t it be fun to identify them?

Now, while the idea is simple, the analysis turned out to be unexpectedly hard and weird. But if you like stats and data science, bear with us.

Calculating pick variance

To find controversial cards, we looked at the average pick number within the draft at which each card was taken, the same way we did to compare card rankings for previous sets.

Numbers in a data set can have a small mean (average) value, or a large one. They may also have small variance (be huddled together), or high variance (be spread wide).

But this time, we also calculated the variance of when each card was taken. For example, if a card is always picked first, its mean pick order would be 1, and the variance would be 0. If the card is picked 2nd in 50% of cases, and 3rd in 50% of cases, the average pick order would be 2.5, and the variance, for large sample sizes, would approach 0.25 (var([2 3 2 3…]) ~ (0.5)^2). We calculated these means and variances for every card, and plotted the variances against the mean, which gave us this graph:

As you can see from the legend, the color scheme is new here: black is for commons, gray for uncommons, gold for rares, and red for mythics.

The cards in the bottom left corner are the awesome mythics and rares, like Doom Whisperer, Dream Eater, and Aurelia, Exemplar of Justice. Everybody likes them, everybody takes them the moment they see them, so they have an average pick order of 1, and a variance of 0 (everybody agrees that they are the first cards to be picked).

On the other end of the spectrum there are points on the very right that represent universally unloved commons: Pause for Reflection, Vicious Rumors, and all five lockets. These cards are always picked last — or almost last — and people more or less agree that it is appropriate to pick them near-last.

Between these two extremes, we have a nice gentle curve. Cards in the middle have higher variance — people feel less strongly about drafting them earlier or later. They are more situational and are also dependent on the guild or deck type the drafter is shooting for. As a result, the more “average” the card is, the higher variance of drafting order it has. Which gives you this pretty hill shape, with the highest variance around the middle.

But then you can notice that there is some extra structure to the hill-shaped plot. Notice for example that commons form a band, uncommons sit on top of this band, and rares are pushed to the upper edge of the uncommons. Why is that?

Before we discuss which cards caused variance among humans, we first need to see where variance occurs naturally.

Inherent draft variance

At first we thought that it may be an artifact of human psychology, but then realized that the stratification appears in this plot even if we analyze decks drafted by Draftsim bots (below). That means that it has something to do with a card’s power, color, and its relative frequency, as these are the only three things our bots “know.”

So, where do these bands come from? They seem to be a byproduct of the drafting dynamics themselves. Imagine a card that is reasonably good, but narrow and demanding: something like a multicolored rare, or the very mono-colored card Gigantosaurus from M19. A player who opens this card in a booster would rarely pick it first, as it’s not good enough to commit to a guild or color. Such a card would get passed around the table, until it hits a player who is already drafting this color or guild anyway, at which point they’ll probably take it.

For these narrow, situational cards, that are decent in some decks, but horrible in others, a mediocre average score comes from a mix of early on-guild (or on-color) and late off-guild (or off-color) picks. These are “cards of opportunity”, that either ask a lot out of your deck (like Thousand-Year Storm), or just have a very limited application (like Drowned Secrets). As an example, here’s a histogram of pick frequency for one of the most controversial cards in GRN, Ionize:

Here’s a practical scenario for you that is hopefully easy to relate to. Imagine that 8 people sit at a table to draft. They open 8 boosters, and one of the boosters has Ionize (a high-variance card) as the pack’s best card in Izzet. The card’s two-colored casting cost (and solid-but-not-amazing power level) makes it not good enough to commit to Izzet right away. There are almost certainly more flexible or powerful cards in the booster, so the player that opens it will probably pass it to the next person to the left.

That person will look at the Ionize, and if they are drafting Izzet, they will be happy to take it. But if they are drafting anything else, they will pass it along, because this card has a low chance of fitting their deck. So it will move across the table until it finds a player drafting Izzet, or until it is drafted as an off-color card towards the end of the booster, because no better options are available.

There is almost no competition for these cards, which leads to this flat histogram, and to a cliff-like decrease in probability of being drafted between the 8th pick (almost full cycle) and the 9th pick (beginning of the next cycle). For a card like that to be drafted on the second cycle, someone would have to pass it the first time around, but later firmly settle on Izzet by the time the booster goes around the second time; something that does not happen that often.

For bears, decent commons, and other “so-so” cards, the situation is very different: you never draft them first (they are not good enough for that), and you almost never get to draft them last, as they fit in many decks, and somebody will almost certainly draft them at some point. Which means that if they have an average score, that’s because they are actually average!

These cards are competitively drafted by many players who compare them to other on-color cards in each booster. As the outcome of these comparisons varies from draft to draft, the histogram of pick orders looks vaguely normal-ish (the Central Limit Theorem starts to kick in), centered somewhere in the middle of the rank scale.

For example, here’s a histogram for one of the least controversial cards with about the same average pick order as Ionize (around 7th), Fresh-Faced Recruit:

So Wizards R&D prints more situational and complex cards at rare or uncommon, creating these bands of gray and gold above the curve of black (commons).

This is our best explanation for the fancy Var/Mean curve, and if you have a better one, please let us know in the comments! The effect is real, and has nothing to do with psychology, but it is also tricky, and hard to wrap your mind around.

OK – here’s the list

But enough with all this talk; what are the most controversial cards? Taking into account the baked-in draft dynamics we just described, the list of controversial cards consists of medium power mythics and rares, mostly two-color, that are situationally good. With raw pick variances, the cards score high on this “controversy” scale not because they polarize drafters necessarily, but because of the logic of the drafting “game.” For GRN, the most “high-variance” or “volatile” cards are:

No controvesry here… — No controversy here…

The least controversial cards, on this scale, are:

All the top choices from the bottom left corner of the graph: awesome, flexible bombs. In GRN, it would be cards like Light of the Legion, Doom Whisperer, Niv-Mizzet, Parun, Status/Statue, Aurelia, Exemplar of Justice, etc.
Bottom-feeders and sideboard cards that everyone either universally hates or knows they can get late: Pause for Reflection, all 5 lockets, Vicious Rumors, Crushing Canopy, Never Happened, etc.
Run-of-the-mill commons (see below for a list).

Human-induced variance

Unsatisfied with this scale, we kept thinking: can we still somehow look past the drafting mechanics and quantify the uniquely human contribution to variance?

How do you find results that do not stem from permutations and distributions, but that are driven by psychology and differences in human opinion?

We realized that we can actually estimate that, because we track both human drafts and bot drafts! All effects of stats, distributions, and relative scores should be the same for bots and humans (except that the ratings used by the bots, and average opinions used by human players, would of course be slightly different).

But humans would have some additional personal variability added to their actions. But bots always use the same ranking system, carefully assigned by Dan Troha, so we can estimate human volatility by subtracting bot pick variability from human pick variability! Here’s the result of this analysis for GRN:

On this plot, the horizontal axis shows how much earlier humans drafted each card, compared to bots. If on average humans drafted something 2 picks earlier, it would have an X of 2 here. The vertical axis shows just how less certain humans were about this card, in terms of bringing extra variance to their pick order.

For example, judging from the yellow cloud on the right side of this square, humans tend to rare-draft (humans pick junk rares and mythics 2-3 picks earlier than we recommend our bot do it). They are also more variable in these preferences, probably because some users raredraft and some don’t. In fact, there’s been a long-running debate among Draftsim users about whether the bots should raredraft because many people do in “real life” scenarios.

And so, finally, here are the most controversial cards, the points in the right top quadrant:

As you can see, most cards that are hard to incorporate in a deck because they are situational and narrow (as we described above) were also confusing for human players. In other words, these are the weird rares.

Which, in hindsight, makes lots of sense. When we are faced with tough decisions, we each solve them differently. That’s what is nice about humans: some like to take risks, some are careful; some stick to strategies that worked in the past, some like to experiment.

So when cards were hard to fit in a deck, this “volatility” was further amplified by human psychology, making them “controversial” not just in a mechanical sense, but also in the psychological sense of this word.

We then looked at the most controversial commons:

Now, this is interesting! In raw, absolute numbers, the five lockets were relatively less controversial than most other commons (remember, they all sit in the right bottom corner of the inverted parabola). Yet if you look past the systemic variance into human-specific variance, you actually see thathuman players had different strategies here. Some were drafting lockets, and some were not!

People love to experiment when they use a draft simulator, and drafting a “sweet” four or five-color locket deck is a nice thing to try out. People were also fairly uncertain about where the lockets fell as far as power level at the beginning of the format.

For the sake of completeness, here are the most controversial uncommons:

A nerdy footnote on mathematical paradoxes

Now, here is an even more mathy aside, for those of you who like to read Frank Karsten and Felix Weidemann for fun. Did you notice that the plot of human-controversial cards actually looks a bit weird, as humans seem to draft virtually all cards earlier than bots do? Look at it again: most of the points have positive X values, meaning that they are drafted by humans a few picks earlier (anything between a fraction of a pick, to 5 picks for Divine Visitation).

When we saw this plot for the first time, we thought that it may be a bug in the code, because how could it possibly be that humans would draft all cards earlier? Surely if they draft some cards earlier, they would draft some other cards later, right? The average values should be the same for humans and bots, as they are all in the same boat, subject to the same rules, wouldn’t you agree?

It turns out that it is actually not a bug, but rather a real full-blown paradox, directly related to Simpson’s Paradox (check out the UC Berkeley admissions controversy, which is closely related to our problem). It turns out that it is possible for one player to draft all or almost all cards earlier, on average, than another player. It is actually a sign of inefficiency. Humans are inefficient when compared to bots — but we knew that!

If you find this counterintuitive and hard to believe, here’s a small trinket example. Say two players are drafting from boosters of 4 cards, and there are only two types of cards, there, A and B. A is more powerful than B. They draft 2 times, and get the following final draft pools:

Draft 1: Player 1: AB, Player 2: AB
Draft 2: Player 1 BB, Player 2: AA

Let’s calculate average pick order for both cards and both players. For Player 1 here, the pick order of A is 1 (it was always drafted first). The pick order of B for the first player is (2+1+2)/3 = 1.7. For the second player, the average pick order of A is (1+1+2)/3 = 1.3, and the pick order of B is 2/1 = 2.

If you compare these average picks, the second player, on average, drafts both cards, card A and card B, later than player 1 does (because 1.3 > 1, and 2 > 1.7). And also, incidentally, they win on average, as AB against AB is a draw, but AA wins over BB.

So, surprisingly, the “Unique Human Preference” plot above not only illustrates some features of human psychology, such as raredrafting, but also testifies to the average inefficiency of human drafts compared to bots. While humans are hunting for fancy cards, bots calmly draft more efficient cards, resulting in more consistent draft pools, and most probably, more powerful decks overall (although we didn’t try to directly estimate that).

Concluding Thoughts

Given that one of the big reasons that you like to use Draftsim is to get better and learn how to improve at drafting, what can you take away strategically from all this?

If you want a quick idea of the format, I highly suggest looking closely at the mono-colored cards on the synergy plot with all the cards labeled to see what guilds they are most closely associated with. This will give you probably the quickest overview of “how to draft the set” because you’ll see which cards work best in conjunction with which other ones.

Secondly, talk to your friends and people you play with about the list of controversial cards. Have they put them to good use? What kinds of decks want them? When can you expect to get them in the pack and how early do you need to draft them? That way when you’re at your next FNM, you’ll know whether to take that weird rare or whether you should just pass it on to the next person.

**(Unless you are a raredrafting human, that is!)

If you enjoyed this article, be sure to follow Draftsim on Facebook and Twitter and please help out the site via Patreon if you can.

Draft Data Analysis 2: How Much Does a Format Change in One Month? [M19]

September 17, 2018 By Dan Troha 1 Comment

By Dan Troha, Arseny Khakhalin, and Bobby Mills

As we finally say goodbye to M19 and welcome in Guilds of Ravnica (appearing on Draftsim.com this Wednesday), our team at Draftsim wanted to take a final look at Magic Core 2019 to see what we can learn from this boatload of data we collected.

This time around we’re going to look at how M19 has evolved since July. Have you ever been able to see an actual list of cards that you over/underrated in draft? We’ll be going through that and a lot more. I have to admit, I was amazed how much the data resonated with my own understanding of the format as time passed.

Plus, there will of course be many pretty new visualizations.

M19 – One Last Time

matrix graphic — It’s time to re-enter The Matrix (matrix math, that is)

Just to recap from our previous article, we visually represented the past three formats with a massive statistical analysis. We collected the picks from over 100,000 M19 drafts in July, then another 50,000+ in August. Then we slice and diced the results in a bunch of different ways, but we mainly looked for pairwise synergies of different cards to find out how they clustered together when people repeatedly picked them for the same decks.

And that analysis produced some sweet plots that looked like these:

2D plot with dots representing cards — M19 August 2018

Note: The units on the axes do not mean anything. They are only there to show the relative relationships between points. For a further explanation, please check out our last article.

We had quite a few people ask us for the data — which was really exciting, but also was kind of overwhelming. It’s clear that the Magic community is awesome and is just as nerdy as we are! We’re planning on releasing this data eventually, but we need to do a lot more work and analysis with it before we can turn it loose on the internet. So stay tuned on that front.

We also received several requests and suggestions for interactive visualizations, and I’m happy to say they are here! Here’s a D3.js version of the 2D visualization above (click on the image):

m19 interactive visualization screenshot — Click here to play with the interactive visualization

But enough of this snapshot stuff. This time around, we’re going to look at … ch-ch-changes.

With a data set this large, we are able to look at relative changes between July and August in card-by-card pick orders and even in overall color pair preferences. For example, we might be able to make guesses at questions like “Which colors became more popular in August?” or “Which archetypes fell out of favor?”

And how could you adjust your drafting strategy if you knew this?

Format Migration – “Flocking”

Let’s first start with the graphic that we teased in the last article:

m19 shown with before and after card movements — “Flocking” of relative changes in card preferences. Click here for a labeled version

The plot above shows every card in the M19 draft format and where it stood before August and after. The large squares (the “heavy heads”) represent the new position, while smaller “tails” represent where each card started in July.

To build this plot we projected July and August drafts onto a 2D picture, and then found a rotation of one picture that matched the positions of cards in the second picture as well as possible (on average). Then we looked at the relative movement of each card within the full cloud.

The most striking thing is that cards mostly seem to move in big flocks! While there are some dramatic individual card changes, it really looks like most of the movement comes from overall changes in the color, not just specific cards here and there. Why would that be?

Overall, it could be that archetypes are becoming more clearly defined and the colors are becoming slightly more polarized. As a format develops during the initial weeks, everyone goes through a learning curve in an attempt to discover what each color combination is trying to do, which color combinations support good archetypes, and which colors are underwhelming when paired with other colors. The flocking shows the format becoming less homogeneous and more clearly defined.

For example, take a holistic look at the color green. The majority of cards appear to be moving in the opposite direction away from the black/white cluster. If you think about what archetypes are represented here — black/green nothing and white/green auras, it certainly could give us some clues.

It is reasonable that players would be less inclined to pair green with black because the deck has somewhat of a bad rap for having no synergy — and therefore is harder to “make work” in a draft. As for white, GW’s primary strength and game plan relies upon an auras theme that takes very specific cards to build. Although green is often trying to ramp in M19, a GW deck is much less interested in Colossal Dreadmaw, Elvish Rejuvenator, or even Druid of the Cowl.

Red, on the other hand, appears to be biasing away from green and slightly toward the white/black cluster. This also would indicate GR losing some popularity (one fun exception is that Onakke Ogre took a hard turn up toward GR). As we’ll see in our archetype and individual card analysis, RW certainly got more popular, and you can see Trumpet Blast, Dark-Dweller Oracle, and Goblin Instigator moving in this direction.

Note: Let me take a moment to say that I also updated the ratings in the system a couple times during this period, which affects which cards are presented to people and the order they are drafted. Not only would the bots take the cards/colors that I bumped up higher, players using the “Suggestions” function of Draftsim would also incorporate the same biases.

screenshot of draftsim's suggestion engine — Draftsim’s AI provides pick suggestions to users — which probably influences some picks

Color Preferences Changed… A Little

Okay, so now that we’ve taken a look at the format as a whole, let’s drill down a little bit and look at each color individually compared to its counterparts.

The tiles below show the share of decks drafted in each pair of colors. The x-axis, “Color 1,” represents the primary deck color — the color with the most cards drafted at the end of the draft. The y-axis represents the color with the second-highest number of cards in the drafter’s final pool.

july 2018 color distribution chart — July 2018 M19 color distribution

In the first pass at the format, we see a strong preference for UW, BW, UB, and GR. Remember, the second color here is not necessarily the color that was drafted second, but rather the color that was second in size in the final draft pile.

UW artifacts and BW lifegain in particular seemed to be pretty obvious synergies right out of the gate. The gold cards told you what to do, there were strong-looking uncommons that supported those themes, and plenty of commons available that acted as enablers. So it’s not too surprising people would start by drafting these decks.

In July, the most popular draft combination contained lots of blue cards and a handful of white cards. But overall, people were drafting most of the color pairs quite diversely. Only WG got no love.

Now let’s compare that to August…

august color data chart — August 2018 M19 color distribution

It seems that after a month of practice, fine-tuning, and mulling over the metagame, drafting strategies started to pull further toward a handful of popular color pairs. BW stayed dominant, RW joined it, while UW, UB, and GR shifted around and became less focused on their primary colors.

Essentially, in August drafters converged on a stronger preference:

1. Draft black, red, or white
2. If the first color isn’t white, draft white for support. A whopping 27% of decks had white as the secondary color. Look at how dark the bottom row on the visualization is!

A special note about white in M19

White was heavily represented as a secondary color. If I had to bet, I would guess this was because of increased discussion in the community about how great white’s commons were, which made people a little more willing to look at white cards for their decks.

Not only does white have a premium removal spell at common in Luminous Bonds, it also has three very strong creatures headlining its slate of top commons. Each of these cards effectively puts the screws on an opponent, and together they are almost unbeatable in their uninteractivity.

Credit to Lords of Limited for dubbing this band the ~~“Three Amigos.”~~ (edit: Three Musketeers, thanks guys).

Hot Decks, Cold Decks

Now let’s give our flocking chart another look, comparing it to the heatmap of changes in color pair popularity, shown on the right.

Here red cells show pairs that became more popular, and blue cells show those that were now drafted less often.

If you let your eye drift up the column for each primary color, you can see that as time passed, Draftsim users really tried to force white, black, and red as primary colors, while they practiced drafting blue and green a bit less frequently. And also, they became increasingly interested in drafting both WR and RW combinations.

Do these observations match the flocking picture? Sort of. The green flock moved away from the other colors, as primarily green decks became less popular overall. Both WR and RW became more popular, so white flock moved a bit towards the red flock.

But overall, the flocking chart does not provide crystal-clear results. Therefore, it might be a little less useful than we hoped. Large-scale movements of colors obscure the individual changes in card-to-card synergies (if these changes exist), and even large-scale movements are hard to interpret. For example, what’s the deal with some of the seemingly-conflicting rearrangements of the blue swarm? Tezzeret only knows.

Still, we tried to look into the movement of individual cards (click here for a full 2000×2000 px version). It’s pretty obvious from the picture that multicolor cards (magenta) were the ones to move the most, which is fun, but reasoning for the movement isn’t obvious.

For example, a whole bunch of cards in the upper left quadrant moved towards green, including Draconic Disciple, Vaevictis Asmadi, the Dire, and Poison-Tip Archer, perhaps implying that you need to be more green than the other color to make use of these cards. This is a phenomenon that you see sometimes in draft — because of the mana or what a particular strategy is trying to do, a gold card can be better in a deck that has a stronger base in a certain color.

Brawl-Bash Ogre and Heroic Reinforcements each move in the same direction as their black and white flocks respectively, while staying equidistant from red, which makes sense.

Ratings Changes

Now that we’ve discussed final color configurations and changes in card synergies, let’s move on to individual card ranks.

Were there any changes in pick order in August compared to July?

Of course there were some changes, but they weren’t overwhelming. Exciting cards were still picked early in the draft (lower left corner), while weak and weird cards were picked late (top right corner). Still there was some curious reshuffling of individual picks, and with our sample sizes (from 1031 for the rarest card to 24733 for most common) we hoped to suss out some real changes, even if they were subtle.

We looked at the top 20% of cards that either lost or gained popularity between July and August. Here’s the data for the most volatile cards, with more popular cards (those that were picked early) shown closer to the top, and less popular cards shown at the bottom. Generally, these popularity rankings correspond to the power level of the card (but not always).

Lines show how much the rank of each card changed. If a card became less popular, the line is blue. If a card’s popularity increased, then the line is red. The intensity of each line represents the absolute change in popularity.

You can read the changes from left to right, where the card name represents the ranking in August after the changes.

So many cards, so many stories to tell. I’m going to dive in by categorizing these in three groups: “subtly powerful cards,” “weird cards,” and “synergy cards.”

Subtle power and format context

By now everyone knows how good Dryad Greenseeker is. But that wasn’t completely obvious at the beginning of the format. It was slept on a little until people got to see it for the card-advantage-machine-with-good-stats that it really is — and now it’s considered a “mythic uncommon.”

The same goes for Departed Deckhand. In a format without many good two-drops, this was one of the absolute best. You could play it in almost any deck, it was good on turn 2, and great on turn 8. The fact that you could get such an impactful two-mana card made it very important to prioritize because there just aren’t that many cards in the format that can provide that amount of impact for the price.

And if there aren’t many good twos, you shouldn’t be putting the bad twos in your deck. But then how do you not die to your opponent’s bad two drops? Essence Scatter quickly rose in stock in M19 because it was a great early play, but unlike a 2/1 for 2, it’s also great on turn 5-6 when you can nab a Thornhide Wolves or Colossal Dreadmaw.

Reclamation Sage experienced the biggest jump, and this is one I’m not completely sure about. Certainly between the existence of a WU artifacts deck, some powerful enchantment-based removal, and gamebreaking bombs (Patient Rebuilding, Sarkhan’s Unsealing) it is quite maindeckable. But there isn’t a huge reason for it to be so relatively underrated initially.

Finally I have to give a shout out to one of my favorites, Vampire Neonate, which had to be one of the most annoying cards to play against in the format. It blanked many of the aggressive two drops and allowed control decks and slower midrange decks to climb back into the game after falling behind. And it was a good mana sync. Not to mention having “the combo” with Ajani’s Pridemate. All for an unassuming one drop 0/3.

Weird cards

Transmogrifying Wand kills some stuff and makes 2/4 oxen. That’s not amazing, right? But wow was this thing powerful. It turns out that if you’re a control deck, or playing against someone who has green creatures in their deck, the card would dominate the game and 3-for-1 your opponent by neutralizing their best threats or even let you upgrade your own creatures.

Inferno Hellion initially looks like a pretty bad card, but there are just enough weird things you can do with it like Thudding or Sarkhan’s Unsealing that it turned out to be quite a bit more playable.

Synergy cards

The presence of a card like Elvish Clancaller (or Goblin Trashmaster) makes you assume there are goblin and elf decks in the format. Unfortunately, M19 did not have these, so they turned out to be much worse in context. Same for Desecrated Tomb — it looks like the card should have ways to trigger it, but there just were not many options in M19 limited, and so it proved to be a very narrow — basically unplayable — card.

On the upswing, you can see that Angel of the Dawn and Heroic Reinforcements jumped to near the top. Word got out (beginning probably with Ben Stark) about how good RW was, and these cards that are a tremendous payoff for a go-wide strategy as it rose in stock.

Concluding Thoughts

You hear a lot about the collective Magic hive mind working to “solve” a standard environment, but the same scale of testing and information sharing goes on with limited. You can see that people combine their own experiences with those of influencers in the community and change their behavior accordingly.

Speaking of, during the first few weeks of a format, I love to hang out on Twitter and look through 3-0 draft decklists to see what’s working for people. Feel free to tweet at me if you discover something cool (or super uncool) about a card or archetype.

Like all of you, I can’t wait for Guilds of Ravnica to drop soon. This is going to mean a whole shiny new data set and some pretty new visualizations of a completely different and guild-based format. I can’t wait to see what a guild synergy clustering visualization looks like…

Oh yeah, and I guess there’ll be a sweet new Magic set to actually play with too!

Be sure to follow Draftsim on Twitter and Facebook to be notified when GRN goes live and when we post more in-depth articles like this.

Draft Data Analysis: What We Learned from Analyzing Over 200,000 Draftsim Drafts

September 3, 2018 By Dan Troha 18 Comments

Hi! I’m Dan, and I run Draftsim, a website that helps people practice by drafting in a simulated environment against seven “bots” programmed with artificial intelligence. Along with some key collaborators, I’ve been working on a big project this year that I’m excited to finally share with you.

Have you ever wondered what a draft format looks like in one picture? Or what makes a format really “synergy dependent”? You’re in luck!

This past year we’ve been collecting anonymous data from the thousands of drafts that occur each day on Draftsim. It’s finally time to take a look at what we can learn from all that sweet, sweet data.

I owe all the credit for this project to Arseny Khakhalin (thank you Reddit for helping us connect!) and Bobby Mills from Bard College in New York. They helped me collect all the data on the site, do the analysis, and create these awesome visualizations.

Methodology

How much data are we talking about? Well, a lot.

Here’s how many human drafts we analyzed for each format:

M19 Part I – 108,000 drafts (July 2018)
M19 Part II – 48,000 drafts (August 2018)
DOM – 51,000 drafts (April 2018)
RIX (including XLN) – 10,000 drafts (February 2018)

In all, that’s actually over 200,000 drafts!

We simply captured the format being drafted, the cards presented to the user in the pack, and the card the user picked given his or her current card pool.

Animated gif of someone drafting M19 — We recorded thousands upon thousands of anonymous user picks

For each draft, we looked at the final pile of cards a user ended up with, and dutifully marked which cards were drafted together in a table that reflected all possible “card pairs.” After going through thousands of drafts, we compared these co-occurrences of cards in final draft pools to numbers that you could expect to get by chance, as if all drafts were completely random. Obviously, some pairs of cards were drafted together way more often than you’d expect by random chance, while some cards were drafted together less often.

At this point we used a standard statistical technique called multidimensional scaling to visualize the allegiances of different cards in a nice plot. To do so, we imagined that each card is represented by a point, and distances between these points depend on how often they are drafted together. Cards with a synergy were assigned small distances between them, while cards that are only drafted together by chance were assigned large distances.

We then utilized a function in the statistical computing language R called cmdscale (the whole analysis was performed in R), and asked it to find an arrangement of points in a 2D or 3D space that would achieve these predefined pairwise distances as well as it possibly could. The function finds an arrangement of points, and then we visualize it either with ggplot (for 2D plots), or with rgl package (for 3D), to get the pictures you’ll see below.

Results and Analysis

So enough with the talk about how we got there – let’s see what we found out! We’ll start with the recent M19 data, mostly from August.

Visualizing M19

On this plot, each point represents a card. Red cards are red, green ones are green, and so on, except that the white cards are gold colored. Colorless and artifact cards are gray. All multicolored cards, regardless of guild, are shown in purple for simplicity’s sake.

If two cards are often drafted together, they are shown close to each other on this plot. If the cards are almost never drafted together, they will be very far from each other on the plot.

Note also that the X and Y axes are not really interpretable; they are just a way to plot the points, but there is no deep meaning to them.

If you’re wondering which card each dot represents, a similar picture that has each point labeled can be found here.

As you can see, it is not easy to represent a complex drafting landscape in a 2D image, so a card may be in the center both if it is drafted with any color (like most artifact cards), and also if it is drafted in decks of two opposing colors (like Gruul or Izzet on this plot). But at least you can spot things that are suspicious and investigate them.

Plotting in 3D might be more informative, but the trouble with 3D plots is that you still need to show them on 2D screens, which really limits their utility. Here’s a 3D projection of M19 drafts: pretty, but hard to use.

a rotating cube with x y z axes depicted relative M19 card positions — M19 live and in 3D

Observations

The first thing we can see here is that cards of the same color are drafted together, which seems obvious. But even then, exceptions prove the rule, and we have quite a few interesting exceptions.

For one, the black and white cards (represented by golden dots) don’t form distinct clusters, which hints that in M19 drafts, players tend to pair W with B more often than with any other color, and vice versa.

The reality is a bit more complicated: white and black cards actually do form distinct clusters, but these clusters are closer to each other than any other two color clusters. So when the multidimensional scaling tried to show all cards on a 2D picture, and looked for some complexity to sacrifice, it sort of folded them together. But if you look at the 3D graph earlier, you’ll see that the white and black clusters are actually distinct. This is an artifact of the 2D visualization method, but it is an artifact that appeared for a reason.

We can also hunt for individual points that don’t belong to the cloud. See that red dot on the very left, far from the main red cluster? That’s Sarkhan’s Unsealing, a card that benefits from large, beefy creatures, and even though it is strictly red, players treated it as a Gruul (RG) card. Or look at that green card that is closer to artifact cards than to other green ones: that’s Scapeshift — which is completely unplayable, however some Draftsim users like to raredraft, so it was drafted as if it were color agnostic.

And while we’re on the subject of unplayables, you’ll also find Omniscience and Mistcaller sitting in the middle since they wouldn’t reasonably be included in any M19 draft deck either.

What about that lonely white card on the far right? It is Aethershield Artificer, which pumps artifact creatures, making it an obvious synergy payoff for the blue/white Artifacts Matter deck.

Enough about relationships, what about power?

We can also mine draft results for other, more straightforward types of information. For example, which cards are the absolute winners that are always picked first? Here’s the top five list:

This is interesting. While these are mostly great pack-one-pick-ones, they’re certainly not what I have rated as the top five cards in the set. Sarkhan especially is strange, but people love planeswalkers and the card is deceptively worse than it looks.

And what cards don’t see any love from drafters? Here are the bottom five, most frequently picked last (so shown backwards, from worst to slightly less worse):

Apparently, “tower” cards are bad.

RIX/XLN visualization

Do all draft formats have similar shapes? Not at all, and of course the Ixalan and Rivals of Ixalan formats are special here, because their synergies are based largely not only on color pairs, but also on tribes that are in those colors. So the plot ends up looking completely different:

Click here for version with cards labeled

Here we have Vampires on the left (with Anointed Deacon and multicolored Legion Lieutenant leading the charge), Merfolk on the right, and the red guys on the top — Pirates and Dinosaurs alike. What is the half-ring of multicolored cards surrounding the red cloud? Pirates are on the left (leaning towards the black Pirates), and Dinosaurs on the right (towards green, and to a lesser extent, white creatures).

Look how closely WB Vampires and UG Merfolk are clustered. These are the two tribes in the set that were represented by only two colors, so they end up grouped very closely together.

Contrast that with Grixis Pirates (B/R/U) and to Naya Dinosaurs (G/W/R). The relationships are much more spread apart and less tightly correlated than the two-color tribes.

Red is in a special place because the color “belongs” to neither Vampires nor Merfolk. When you pick a red card, assuming an equal archetype distribution, it has an equal chance of ending up in a tribal deck of any of the four colors — either a Pirate deck (RB or UR) or a Dinosaur deck (GR or RW). In contrast, it’s not so with black, where you can end up in Pirates (RB or UB) or Vampires (WB) only, if you stick to the prescribed tribal archetypes and groupings of synergy.

In XLN/RIX, GB and UW were “tribeless,” making their synergies — if they existed at all — much less pronounced. It makes sense that we don’t see them bumping up against each other here.

DOM visualization

Click here for version with cards labeled

Dominaria seems to have the most separation between colors of all three. Perhaps this has to do with the high individual power level of many of the cards in the set. Moreover, in DOM, artifacts do not form an amorphous cloud in the center, but make a cluster of their own, very close to U/W/B, which happen to be the colors that care much more about historic triggers.

Not too surprisingly, the blue cards that pull closer to red are Wizards synergy cards (eg Naban, Dean of Iteration, Vodalian Arcanist, and Wizard’s Retort), and the ones that are closer to white are either artifact/historic payoffs or enablers (The Antiquities War, Artificer’s Assistant, Zahid, Djinn of the Lamp, Karn’s Temporal Sundering, Sentinel of the Pearl Trident).

I’m not sure how to explain why white is so closely clustered together versus red, however. This possibly indicates a replacability/lack of synergy among white’s cards or perhaps a tendency of the white cards to only be drafted with each other.

Bonus: Just for fun, here are the other two 3D visualizations that we generated for RIX and DOM:

a rotating cube with x y z axes depicted relative RIX/XLN card positions — RIX 3D visualization

a rotating cube with x y z axes depicted relative DOM card positions — DOM 3D visualization

Where To Go From Here? Some Exciting Future Possibilities…

So this is all really cool, but what can we do with it?

Training Draftsim’s AI to use this data and capture synergy

Synergy in Magic is a very difficult thing to measure because it shifts from format to format. In one block, R&D may introduce a new mechanic that explicitly drives synergy (such as historic, kicker, or cycling), but another might primarily use tribes, such as in Ixalan. And let’s not even get started with cards that are “better for aggro decks” or “control” decks. After all, most decks in limited are just some flavor of midrange, right?

Rather than individually scoring or programming all these factors for each set, it would make a lot of sense to enlist Draftsim’s wonderful user base to help train the algorithm to recognize synergistic pairings of cards.

We could then in turn train the AI to make more realistic decisions based on the clusters of cards that are frequently drafted together.

Encapsulate the shifting “draft metagame”

It would be fascinating to track how cards and color pairs become progressively more or less overvalued during the course of the format. You might see certain color pairs move around as archetypes go in and out of vogue or one color gets recognized as “bad.” And, like in constructed, a draft metagame seems like it would eventually settle somewhere.

And wouldn’t it be really fun to be able to quantitatively measure what cards people “slept on” at the beginning of a format? Or to assign archetypes and see how our understanding of what cards belong in them changes? You could then use this information to improve your card and archetype evaluation methods.

In fact, this leads into our next article where we’ll do a quantitative analysis of how much a format changes over the course of a month. Here’s a little taste of what we’ll be looking at:

That’s right – we have before and after data that we can use to measure changes. Double the data means double the fun!

double rainbow guy saying double data — Stay tuned to find out…

Clustering decks instead of cards

This whole article we’ve been looking at defining a format by the relationships between individual cards. However, we can also zoom out and examine the the end product of your draft as well: your final collection of drafted cards.

By looking at the features of these collections, we can use unsupervised learning algorithms to group decks into archetypes. This will give us a clear picture of the kinds of decks that you will likely face in a draft metagame, the key cards in those decks, and frequency of those decks being drafted.

Not only would this help all of us learn a ton about drafting, we could start exploring draft bot “personalities.” Maybe one bot at your table is leaning towards drafting something aggressive, while another bot just can’t wait to Cancel every bomb you draft. These personalities could very well mimic the same drafting preferences you see from people at your local game store.

Final Thoughts

One thing that really surprised me was the shapes of the formats — even in the more meaningful 3D visualizations. RIX looks incredibly different than the more “normal” M19 and Dominaria formats. Perhaps by looking at these shapes and patterns, we can quickly figure out some important or valuable points about approaching a format. As as we get more data from different sets, more patterns should emerge.

This is such a treasure trove of data that I’m sure there are many ways we could use this methodology to learn more about Magic and about its different draft formats. Let us know what other awesome things you think we could do or what else you’d like to see measured and analyzed!

If you enjoyed the read, be sure to follow Draftsim on Twitter or Facebook for more. And why not give the draft simulator a try?