Fantastic Blognanza!

The question for this edition of Threadless Numbers, which will likely be the last for the foreseeable future, comes from MJ at Compete-tee-tion, and I thought it was a great one and quite a reasonable extension of the first Threadless Numbers. Let’s get on with it, then!

What’s at StakeMore comments correlate with a higher score; could either getting more comments or a higher score (or both) be because the designer has had their submissions printed before (i.e. is an ‘alumnus/a’)?

Where Are the Numbers?Sadly, nowhere. Okay, not nowhere… you can find them on the Threadless website. But I don’t have them. That isn’t going to stop me from talking a bit about this problem, though! I’m going to talk about the question, problems with it, and then lay out a specific project that would serve as a good beginning for anybody who might want to follow up on my research.

Spuriousness and Direction of CausalityThere are some problems with trying to show that a design got either a lot of comments or a high score because they were a Threadless alum. It could be the case that a submission gets a higher score purely because the designer has a reputation of being ‘good’ (as defined by the judgment of the Threadless community) because they’ve been printed before. (I’m dropping number of comments from this thought experiment because it’s simpler.) This story seems plausible. It might seem reasonable, then, to compare the scores of submissions from alum designers to those of unprinted designers. But there’s a glaringly huge problem with that: alum designers might have a higher average score not because they’ve had submissions printed in the past, but because their designs are ‘better‘ than those from unprinted designers. In other words, the correlation (if there is one) would be spurious, or caused by a third variable.

Let’s Try to Test It Anyway!I wish we could! But here’s the thing: to control for the ‘goodness’ of a submission we would have to quantify it. Besides the design’s average score (which is one of the variables we’re already trying to use in our correlation), how can we? We can’t use the comments on the sub because they are almost certainly caused by the same thing that causes it to have a higher average score. Unless the same design could be submitted under two users, one an alum and the other not, without the community somehow noticing (which is basically impossible), I can’t think of any other way. There is, however, something else we could do.

What Else Can We Do?Although we can’t control for the ‘goodness’ of a submission, we can still try to look at how Threadless alumnus/a status affects the number of comments on a submission, regardless of how ‘good’ that submission is. Here’s how it would be set up: whether or not the designer of a submission is a Threadless alumnus/a is our categorical independent variable, while the number of comments on their post-alum submissions is our continuous dependent variable. Analysis would take the form of a comparison of means, controlling for the average score a design received.

Where Can I Get the Necessary Data?I’ll tell you where! You’d have to look at designs that have completed scoring. Data for all three of the necessary variables can be obtained from this single page:

the design’s average score (in the right-hand column)

the number of comments the design received (in the right-hand column)

whether or not the submitter is a Threadless alum (look for the little shield either next to a ‘This is my design’ comment or on their profile page)

So what are you waiting for?

Reminisce with ‘Threadless Numbers’
Threadless Numbers #1: Is it possible to predict how well a design submitted to Threadless will score?
Threadless Numbers #2: What’s the difference between professional and amateur designers on Threadless?
Threadless Numbers #3: What kind of t-shirts don’t get printed at Threadless?

Because this feels like a lightweight week to me, I think it deserves a lightweight Threadless Numbers, and what’s more lightweight than losing Threadless t-shirts? But really I kid; there are a lot of good subs that never get printed, which leaves room for wonderful companies like Go Ape and Cotton Werks.

What You Wanna KnowWho wins the t-shirt design contest on Threadless? What are their winning designs like? Well, I do have some information on that sort of thing, but…

What I’m Gonna Tell YouAs of 5/6/2007, out of the most recent 120 submissions that had completed scoring, exactly 100% had been dropped from the scoring process after 24 hours because of the updated 1.5 rule. Basically, designs that aren’t scoring well get dropped after 24 hours. What this means is that most of the submissions to Threadless aren’t liked by the community. Which is why the focus of this ‘Threadless Numbers’ is losing submissions.

Reasons for Disliking DesignsI couldn’t possibly claim to know why every person rates a particular t-shirt design poorly, but there are two large contributing factors that stood out to me in the comments of these designs: 1) the design is too similar to something previously printed by Threadless and 2) the design is too distant from Threadless’ aesthetic.

Too Hot?A case in point for a design too similar to a previous winner is Beware of the Innermost Gnome, which is reminiscent of Nesting Dolls (in that they both use nesting dolls for their gimmick). I wouldn’t be surprised to find that winning designs inspire others to think up designs that are somewhat similar; in this case, I think people probably didn’t like the colorway and naive style of Beware of the Innermost Gnome.

Too Cold?As for something that’s too different from Threadless’ aesthetic… I can’t find a single striking example, but if you’ve spent any time scoring t-shirt designs, I’m sure you’ll know what I’m talking about.

The Final ScoresThe mean score of these submissions that had been dropped from the scoring process is 1.403, ranging from .99 for Skully to 1.72 for King. For comparison, the printed designs I discussed in Threadless Numbers #1 averaged 2.772 (or 1.369 greater than the mean score for dropped designs), ranging from 2.17 for Le Voyage Dans la Lune to 3.77 for Can’t See the Forest but for the Socks.

Comments on Losing SubmissionsRemember how I talked about the positive correlation between submissions’ final average scores and the number of comments they receive during scoring in Threadless Numbers #1: Is it possible to predict how well a design submitted to Threadless will score?? Well, the average losing design receives 11 comments, compared to 91 for the winners. That these losing designs were dropped from the scoring process after 24 hours (or 1/7th the amount of time that the winning designs were scored) must be taken into account, however. An adjusted comparison, then, would be 77 to 91.

A GraphWhat would this post be without a graph? Not much, that’s what. But don’t worry: no statistical analysis!
The number of comments (adjusted for being in the running for only one day) on dropped submissions is pretty random, and has a range almost as large as that for the printed designs. There are a couple things that would have made sense: 1) there would be more comments on subs that are either really good or really bad (or controversial), because somebody is more likely to have something to say about a design they feel strongly about than one for which they have a pretty neutral opinion. But without filling in the 1.7 to 2.2 range of scored submissions (which are generally either score too well to get dropped from the running or too poorly to get printed), it’s hard to say whether that’s the case.

2) Another possibility is the one which I originally proposed, that as the submission’s score increases, the number of comments the sub receives also increases. The correlation almost looks exponential, but having to adjust the number of comments for the dropped designs certainly introduces additional error, and missing a large portion of data is probably skewing the shape of the correlation; so the fit of an exponential curve is pretty bad.

ImplicationsI have no further generalizations to make at this time, save that more research needs to be done to make up for what mine lacks. If anybody wants to talk to me about doing their own research, I’d be more than happy to!

Continue Reading ‘Threadless Numbers’
Threadless Numbers #1: Is it possible to predict how well a design submitted to Threadless will score?
Threadless Numbers #2: What’s the difference between professional and amateur designers on Threadless?
Threadless Numbers #4: Does being a Threadless ‘alumnus/a’ give your submissions an advantage?