If you haven’t read this article about Survivorship Bias (via Toby Buckell), and our tendency to give more weight to successes than failures, go read it. I’ll wait.
Done?
Survivorship bias plays into all sorts of things. In fact, it has a great deal to do with how we teach history, too, which I touched on in my “We Have Always Fought” post (hence we focus on populations we perceive to be successful *now* and ignore ones that actually have a lot of relevance both now and in the past). But that’s not what I’m going to focus on here. Instead, I want to talk about how survivorship bias plays into marketing and advertising strategies – not just because this is what I do all day at the day job and for freelancing clients but also because it’s something I put into practice when marketing my own work – and in creating my own work, too.
I churn out an incredible number of projects every year at my day job. Upwards of 600. That 600 combines direct mail, email, web pages, registration pages, literature pieces, brochures, video scripts, banner ads, and oh so much more. One of the things that happens when you put out this much content every year is that you have a lot more data to work with. Instead of one-shot emails or web pages, you do a lot of a/b testing. You test a lot of subject lines for emails, personalization techniques, and different types of touch combinations(direct mail, then an email, then follow-up call). The nice thing about all that data is that you quickly find out what works – and what doesn’t. And sometimes it comes up 50/50. Which is… less than useful, because, hey, we could have just a flipped a coin and gotten the same results!
The trouble is, in pretty much every industry, we want to focus on the half of the stuff that works and ignore the half that doesn’t. This is a human thing, and has a lot to do with job retention, too. We want to bring our successes to our bosses “Hey, this email got a 60% open rate!” But in fact, what I’ve realized this last year is that I’m learning a lot more from the pieces that fail than I am from the ones that succeed. Successful pieces tend to get used again and again until we wear them out. But the failures are sometimes perpetuated, too, instead of learned from, and that’s always frustrating. Why does this happen? Because instead of figuring out why the failure didn’t work, we often assume it must be a problem with the time it was sent, or maybe all the customers were on vacation, or… some other external not-to-do-with-my-job thing. In truth, the only underperforming campaign of mine I can attribute to an external, non-messaging thing is when an email went out to a bad list. If you’re not sending a message to real people, well – that’s a problem I can’t solve with my magic keyboard here.
The cool thing about having so much data – particularly when it comes to email – is that you can figure out exactly where you lost somebody. If you have a low open rate, it’s probably because your subject line wasn’t compelling. If you have a high open rate but low click-through rate, that means the message or call to action inside your email wasn’t compelling or was difficult to understand. And if you have a good open rate and good click-through rate, but hardly anybody filled out your registration page or e-commerce page you linked to, then the problem is on that landing page. Once you know where the problems are, you can test different types of solutions.
So nobody’s filling out your landing page/buying the product. Can you pre-fill the registration page with their information? Are you losing them once tax and shipping is added? Maybe you should add those costs in up front, or add the equivalent cost to the actual price and then discount them at checkout so it comes out even (I have never noticed a company do this, but I think it’d be interesting – free shipping! is a huge incentive). Are you simply asking them to buy too soon? Can you give them something of value first, like a coupon or whitepaper or free podcast or ebook and then have a salesperson follow-up?
I remember getting handed a series of email projects marketing a product to a really tough audience. When I asked for analytics on previous emails, I discovered we had an abysmally low open rate, click through rates, and conversions. It was just… all bad. It was a tough nut to crack, and nobody had figured it out yet. The first thing I did was ask the client to send me the emails that didn’t work. Why? Because I didn’t want to risk coming up with something – or something similar – to what hadn’t worked before. Madness is doing the same thing over and over again, in the same way, and expecting a different result. But people in marketing do this all the time. Because if something fails and you’re questioned about it, you can always say, “I just did what we always do!” But if something fails and you did something different, you have to say, “Yes, I did something different, and it failed.” And taking that kind of responsibility scares the crap out of people.
I knew right away that we had a couple of problems. And some of those problems were things I had done when I created the original product messaging. I’d basically said the equivalent of, “You’re not very good at your job, but we can make you good at it!” (no, it wasn’t that bad, but you get the idea). This is bad messaging. I’d just read several great blog posts about how flattery is the best form of marketing, and this really failed at doing that. So I fixed the messaging. It went from, “You’re bad at your job but we can make you good at it!” to “You’re amazing at your job, and here’s how you can be even better.”
These performed better, but we still weren’t getting the numbers we wanted. So I looked again at the subject lines, since that’s key to open rates. Subject lines need to convey immediate value/benefit. They need to be very specific to the target audience, and completely relevant. If a subject line isn’t compelling, I don’t open it. Looking at some of the subject lines we had – which were twice as long as standard and included long —- dashes for some reason, I went back to basics. I created a six-word, two sentence subject line.
And, lo, open rates hit 60% (industry average is 20%). 
Next up will be mining the data that comes back. We have better open rates, better click-through rates, and now we need to see how conversions are. If those still suck, guess what we reconfigure next? The conversion page.
But none of this would have changed if I focused only on the successful emails or pieces we’d created for this market. In part because, let’s be real – none of them were very successful. If I’d only mimicked the “most” successful piece, it’d still be performing at that standard. It’d be a constant game of just keeping up with what we were doing now, instead of trying to surpass it. Instead of doing better. The more we focus on “success” the more we focus on the one-offs, the quirks, the outliers. It’s focusing on failures and middle-of-the road pieces that teaches me how to improve.
It’s easiest to apply these sorts of message tweaks to digital marketing, where you have actual data. How does this apply to book marketing? Let’s take a recent test I did with Facebook ads for my book, God’s War. I’m not a fan of Facebook ads because I’m not convinced they work all that well, but because they have good analytics, it’s a nice testing ground for messaging.
When God’s War first came out, I tested a lot of different ads. First I tested subject lines. “Meet your match,” “Save the World,” “Power to the People.” Weirdly enough, “Meet your match” did better. Maybe people thought it was a dating ad?
But giving away something of value: “Get a copy of God’s War” performed even better (as you’d imagine). What if I couldn’t give away free copies? How about reviewer blurbs? Reviewer blurb did OK. But how about blurb with a new cover? Turns out my ad with the new cover of God’s War and a Lauren Beukes blurb outperformed everything but the free book. Trouble is, I didn’t know if it was the Beukes blurb or the new cover that pulled people – next time I’ll run ads simultaneously.
This was an interesting testing ground, but let’s be real about the performance of Facebook ads, because this is part of survivorship bias, too. The best click-through rate I had on a non-sponsored ad was 0.041%. Industry average is 0.1-0.3%. Yes, without the extra 0. So by industry standards, Facebook is, overall, a pretty poor place to put your ads, even after testing and refining ads the way I did. Sometimes, at the end of the day, what you learn from your failures is that it’s your choice of ad placement itself that’s the ultimate failure.
Survivorship bias, however, had a lot of folks believing Facebook was the way to go a few years ago. A couple of marketing firms with outlier ads that performed well would get on their blogs and shout about how Facebook was the next big marketing platform. It wasn’t until recently that folks began to acknowledge that actually, Facebook ads were sort of a waste of time. Hence Facebook’s attempt to switch to sponsored stories and promoted posts, which perform much better (one of mine had a 3% click-through rate. Yes. Integer. Of course, that’s just one ad…).
But what did I learn from the failure of the Facebook ads? I learned that free books and quotes from readers work best. I learned that even though I love my pulpy Night Shade covers to pieces, the Del Rey UK cover performs better. And sponsored stories – stories endorsed by friends, which is a sort of forced word-of-mouth – perform even better.
Now I put all those together, let’s say – the Del Rey UK cover, the Lauren Beukes blurb, and then I put it in a place where ads perform better, let’s say, on Goodreads or a genre magazine site, and ideally, let people’s endorsements from FB or Goodreads show up right there on the ad. So my Goodreads rating is there, or maybe it scrolls through different publicly posted reviews of it, attributed to users. People may not like this type of marketing. When Facebook and G+ added the ability to publicly like stories on websites themselves there was a lot of talk about privacy issues. But let’s face it – stories endorsed by our friends get shared more. Books our friends talk about get read more.
Finding out what failed also makes me better at managing my time. Podcasts, radio, and even tiny little TV spots will sell more books than any ad or anything I could do on my own. Guest blog posts generally don’t sell anything – my recent outlier aside – but me blogging about other people’s books will sell copies of those books. How do I know that? Because I track that with Amazon affiliate links. You know what else sells books? Book buyers at bookstores who like my books. That sells lots. Awards maybe sell a thousand or so more, depending on the award and your already existing profile.
But after you make folks aware of your book – which is really all this trickery with ads is about – whether or not they buy it and recommend it depends entirely on how good it actually is, or whether or not they connect with it, or how successful you are at getting the right people to like it, and thus giving it a symbolic value that goes beyond its real value (think: diamond rings). And achieving those require different types of tactics.
One of my favorite examples is the Dvorak keyboard. We could all be typing twice as fast (or faster) using a Dvorak keyboard. So why don’t we? It provides greater actual value. But people are used to our standard Qwerty. Changing everyone now would take a massive marketing effort. It would take key individuals changing their preferred keyboard, and talking about it. Big public awareness campaigns. Testimonials. It would take creating an Apple-like “you’re smarter than everyone else because you choose us” type of messaging.
Actual value does not equal market value, which I think is something that a lot of writers struggle with. I know I do. Why does this awesome amazing book do less well than this shitty one? Why is everyone reading this crappy been-there, done-that book? We can talk all day about how it’s because it’s “fun” and they “like it” but why do we like things? Why do we buy diamond engagement rings? Why do we drink Coke over Pepsi? Before Tolkien, we weren’t reading Tolkien knock-offs (and most every fantasy book doing well right now is a response to Tolkien, positive or negative). But when we’re given a new standard, when it’s popularized, there’s suddenly a glut of it (think YA. Hordes of writers weren’t fleeing adult lit to write YA before Harry Potter, and let’s not even get into the weird YA dystopian threesome trope that’s been inspired by Twilight and Hunger Games).
I’m not convinced that readers read things because they’re intrinsically good or entertaining. I think something happens when a band wagon comes into town, and all of one’s friends jump on. I think we crave to be part of something; we want to talk with our friends about the media we view. “Oh you have to watch X show!” we tell folks, not only because we think it’s great, but because we want to talk to somebody else about it. We want to debate it. We want to unpack it.
Genre “movements” happen all the time, where you can point back to one book – generally an outlier for the time – that was very successful, and thus inspired a ton of knock-offs. And then suddenly we all get told that really, it’s popular because that’s what people like. They just “like” it. And so we look at that book, and we mimic it. We try to figure out why it was successful.
When instead, what I’m more interested in is looking at books that have all the trappings of success – they’re well written, well-plotted, sensawonder books that just aren’t loved. Why aren’t they loved? Because no one’s heard of them? Why haven’t we heard of them? What inspires love? Is it that the right people, the right influencers, simply didn’t read it? Or didn’t connect with it? And how much control do we have over that, as writers?
What I realized as I started work on my space opera last year is that I didn’t actually want to just write something easily accessible that I knew people would like. If China Mieville had done that… well, yeah. Those aren’t easy books; they’re a bit of a mess. Want I wanted to write, instead, was a category creator. What I wanted was to make something that stood out, and carved its own subgenre and inspired its own imitators.
But I’m not going to get there by just studying the books that broke out. When you study the breakouts, the outliers, you tend to want to follow them. Just like with those successful emails, you’re more likely to try and mimic success instead of learn from failure to create something even more successful. By unpacking beautiful books that didn’t sell well, I’m more likely to identify areas where I can improve – not just the fiction itself but also my business sense, because let’s be real – if readers haven’t heard of a book, or can’t find it, they can’t buy it or recommend it. Hence the importance of doing some marketing up front for new books to raise awareness and ensuring you have a publisher with a distribution system that doesn’t suck. It’s also good to peel back what happened behind the scenes with promotion and packaging. Was the cover turning off readers? Was it targeted to the wrong people?
When you start breaking down the individual failures of a book’s launch, you can look for ways to improve each of those parts for yourself. Content, messaging, promotion, packaging, distribution – they all play a role. But just saying, “Hey, this one book (which may only be successful because it got a couple of those right) is the holy grail and I’ll just focus on those three things it did right!” means you could still fail abysmally at the others.
One of the things survivorship bias has taught me in both my day job and novel writing capacities is that if I focus on the successes, I’m far more likely to come up with something mediocre than if I focus on the failures.
