The question for this edition of Threadless Numbers, which will likely be the last for the foreseeable future, comes from MJ at Compete-tee-tion, and I thought it was a great one and quite a reasonable extension of the first Threadless Numbers. Let’s get on with it, then!
What’s at StakeMore comments correlate with a higher score;
could either getting more comments or a higher score (or both) be because the designer has had their submissions printed before (i.e. is an ‘alumnus/a’)?
Where Are the Numbers?Sadly,
nowhere. Okay, not
nowhere… you can find them on the Threadless website. But
I don’t have them. That isn’t going to stop me from talking a bit about this problem, though! I’m going to talk about the question, problems with it, and then lay out a specific project that would serve as a good beginning for anybody who might want to
follow up on my research.
Spuriousness and Direction of CausalityThere are some problems with trying to show that a design got either
a lot of comments or a
high score because they were a
Threadless alum. It could be the case that a submission gets a higher score purely because the designer has a reputation of being ‘good’ (as defined by the judgment of the Threadless community) because they’ve been printed before. (I’m dropping number of comments from this thought experiment because it’s simpler.) This story seems plausible. It might seem reasonable, then, to
compare the scores of submissions from alum designers to those of unprinted designers. But there’s a
glaringly huge problem with that: alum designers might have a higher average score not because they’ve had submissions printed in the past, but
because their designs are ‘better‘ than those from unprinted designers. In other words, the correlation (if there is one) would be
spurious, or caused by a third variable.
Let’s Try to Test It Anyway!I wish we could! But here’s the thing: to control for the ‘goodness’ of a submission we would have to quantify it. Besides the design’s average score (which is one of the variables we’re already trying to use in our correlation), how can we? We can’t use the comments on the sub because they are almost certainly caused by the same thing that causes it to have a higher average score. Unless the
same design could be submitted under two users, one an alum and the other not, without the community somehow noticing (which is basically impossible), I can’t think of any other way. There is, however, something
else we could do.
What Else Can We Do?Although we can’t control for the ‘goodness’ of a submission, we can still try to look at how Threadless alumnus/a status affects the number of comments on a submission, regardless of how ‘good’ that submission is. Here’s how it would be set up:
whether or not the designer of a submission is a Threadless alumnus/a is our categorical independent variable, while the
number of comments on their post-alum submissions is our continuous dependent variable. Analysis would take the form of a comparison of means, controlling for the average score a design received.
Where Can I Get the Necessary Data?I’ll tell you where! You’d have to look at designs that have
completed scoring. Data for all three of the necessary variables can be obtained from this single page:
the design’s average score (in the right-hand column)
the number of comments the design received (in the right-hand column)
whether or not the submitter is a Threadless alum (look for the little shield either next to a ‘This is my design’ comment or on their profile page)
So what are you waiting for?
Reminisce with ‘Threadless Numbers’
Threadless Numbers #1: Is it possible to predict how well a design submitted to Threadless will score?
Threadless Numbers #2: What’s the difference between professional and amateur designers on Threadless?
Threadless Numbers #3: What kind of t-shirts don’t get printed at Threadless?
Subscribe to our Feed!
Recent Comments